Jul 09 22:06:38.646097 ip-10-0-141-248 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 22:06:38.646107 ip-10-0-141-248 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 22:06:38.646114 ip-10-0-141-248 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 22:06:38.646500 ip-10-0-141-248 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 22:06:48.663111 ip-10-0-141-248 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 22:06:48.663126 ip-10-0-141-248 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2d5e6af35e3f46e491624af367625b6f --
Jul 09 22:08:39.992547 ip-10-0-141-248 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.966446    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971277    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971289    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971294    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971298    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971301    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971305    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971308    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 22:08:41.052827 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971311    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971314    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971317    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971332    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971335    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971339    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971341    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971344    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971347    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971350    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971353    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971357    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971362    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971366    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971371    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971374    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971377    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971380    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971385    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 22:08:41.054148 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971389    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971393    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971396    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971399    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971401    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971404    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971407    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971410    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971413    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971415    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971419    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971422    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971425    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971428    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971431    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971434    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971437    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971439    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971442    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971445    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 22:08:41.058923 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971448    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971450    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971453    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971456    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971458    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971461    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971464    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971467    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971471    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971474    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971477    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971480    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971483    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971486    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971489    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971491    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971494    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971496    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971499    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971501    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 22:08:41.059800 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971504    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971507    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971510    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971512    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971515    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971518    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971523    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971527    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971530    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971533    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971536    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971539    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971542    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971544    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971547    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971550    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971553    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971555    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971560    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 22:08:41.062890 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971563    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971566    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971569    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971572    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971575    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971577    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971580    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971583    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971586    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971588    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971591    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971594    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971597    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.971599    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976732    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976747    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976751    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976755    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976759    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976762    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 22:08:41.063717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976765    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976768    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976772    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976775    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976779    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976782    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976785    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976792    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976795    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976798    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976801    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976805    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976808    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976811    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976814    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976818    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976821    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976824    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976828    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 22:08:41.067222 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976834    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976837    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976840    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976843    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976845    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976848    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976851    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976855    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976857    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976861    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976864    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976867    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976869    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976875    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976878    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976881    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976884    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976889    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976892    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976894    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 22:08:41.068089 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976897    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976900    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976904    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976906    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976909    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976915    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976918    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976921    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976923    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976927    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976929    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976933    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976936    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976939    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976942    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976946    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976950    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976956    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976959    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 22:08:41.072875 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976963    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976967    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976971    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976976    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976979    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976982    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976985    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976989    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.976994    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977000    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977003    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977006    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977009    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977012    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977015    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977018    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977022    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977025    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977028    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 22:08:41.073741 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977032    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977035    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977038    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977043    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977047    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977050    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977053    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977056    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977059    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977062    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977065    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977068    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977071    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977074    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977077    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977080    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977843    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977849    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977853    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977856    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 22:08:41.079023 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977860    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977863    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977866    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977869    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977872    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977881    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977884    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977889    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977893    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977897    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977900    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977904    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977909    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977912    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977915    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977918    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977922    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977928    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977931    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 22:08:41.079906 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977934    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977936    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977939    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977942    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977945    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977948    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977951    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977954    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977956    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977960    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977963    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977968    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977973    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977976    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977980    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977986    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977989    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977992    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.977997    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978000    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 22:08:41.083405 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978003    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978006    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978012    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978015    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978018    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978021    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978024    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978027    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978029    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978033    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978038    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978042    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978046    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978050    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978053    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978059    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978062    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978065    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978067    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 22:08:41.084271 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978070    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978073    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978076    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978079    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978082    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978085    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978087    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978090    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978096    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978098    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978101    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978105    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978110    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978113    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978118    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978120    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978123    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978126    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978129    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978132    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 22:08:41.192005 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978137    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978140    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978143    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978146    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978149    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978152    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978159    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978162    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978165    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978167    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978171    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978174    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978177    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978182    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978185    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978188    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.978191    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978278    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978285    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978296    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978301    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 22:08:41.192849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978305    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978309    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978316    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978337    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978342    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978349    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978353    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978357    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978361    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978364    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978367    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978371    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978374    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978377    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978385    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978388    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978393    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978396    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978400    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978404    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978407    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978411    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978417    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978421    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 22:08:41.197259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978424    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978427    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978430    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978434    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978438    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978442    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978445    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978451    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978454    2566 flags.go:64] FLAG: --enable-server="true"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978458    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978465    2566 flags.go:64] FLAG: --event-burst="100"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978468    2566 flags.go:64] FLAG: --event-qps="50"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978472    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978477    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978480    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978487    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978490    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978494    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978497    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978501    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978504    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978507    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978511    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978514    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978520    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 22:08:41.198135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978524    2566 flags.go:64] FLAG: --feature-gates=""
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978528    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978531    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978534    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978538    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978541    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978545    2566 flags.go:64] FLAG: --help="false"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978551    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978554    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978558    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978561    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978565    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978568    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978571    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978574    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978578    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978583    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978588    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978591    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978595    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978600    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978603    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978606    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978609    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 22:08:41.203829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978615    2566 flags.go:64] FLAG: --lock-file=""
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978618    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978622    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978625    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978631    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978634    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978638    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978641    2566 flags.go:64] FLAG: --logging-format="text"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978644    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978651    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978654    2566 flags.go:64] FLAG: --manifest-url=""
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978657    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978662    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978666    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978670    2566 flags.go:64] FLAG: --max-pods="110"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978673    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978677    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978682    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978686    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978689    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978693    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978696    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978705    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978708    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978714    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 22:08:41.204691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978719    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978722    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978728    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978733    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978737    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978740    2566 flags.go:64] FLAG: --port="10250"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978743    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978749    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07f3ce2749302f63b"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978753    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978759    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978762    2566 flags.go:64] FLAG: --register-node="true"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978765    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978769    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978773    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978776    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978779    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978783    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978932    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978940    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978944    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978947    2566 flags.go:64] FLAG: --runonce="false"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978950    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978954    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978957    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978960    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 22:08:41.205588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978963    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978966    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978970    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978973    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978976    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978979    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978983    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978985    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978990    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.978994    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979002    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979011    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979015    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979018    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979023    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979026    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979029    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979032    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979035    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979038    2566 flags.go:64] FLAG: --v="2"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979043    2566 flags.go:64] FLAG: --version="false"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979047    2566 flags.go:64] FLAG: --vmodule=""
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979051    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.979055    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979183    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 22:08:41.210366 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979187    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979190    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979193    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979196    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979200    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979202    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979206    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979208    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979211    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979214    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979217    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979220    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979223    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979226    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979229    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979233    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979236    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979241    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979244    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 22:08:41.211209 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979246    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979249    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979252    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979263    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979266    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979270    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979273    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979275    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979278    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979281    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979291    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979294    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979297    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979300    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979303    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979306    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979309    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979311    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979314    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979317    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 22:08:41.212061 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979333    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979336    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979339    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979341    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979344    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979347    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979349    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979352    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979357    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979360    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979366    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979368    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979371    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979374    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979377    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979380    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979382    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979386    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979388    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979391    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 22:08:41.215129 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979394    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979397    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979400    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979404    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979407    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979409    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979412    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979415    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979417    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979420    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979423    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979425    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979428    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979431    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979434    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979438    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979442    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979445    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979448    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 22:08:41.216009 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979452    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979458    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979461    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979468    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979472    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979475    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979478    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979481    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979484    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979488    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979491    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979494    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979497    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979500    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979503    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979506    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979509    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979512    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979515    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 22:08:41.324509 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.979518    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.980398    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.987682    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.987700    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.988558    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.988570    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988637    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988643    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988647    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988651    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988655    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988657    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988660    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988663    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988667    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 22:08:41.325317 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988670    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988672    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988675    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988678    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988681    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988684    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988687    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988689    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988693    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988695    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988698    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988701    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988704    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988706    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988709    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988713    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988716    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988719    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988723    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988726    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 22:08:41.328806 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988729    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988732    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988735    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988737    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988740    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988743    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988746    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988749    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988752    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988755    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988757    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988763    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988767    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988770    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988773    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988776    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988779    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988781    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988784    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988787    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 22:08:41.329717 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988790    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988794    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988796    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988799    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988802    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988805    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988807    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988810    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988813    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988816    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988819    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988822    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988825    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988828    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988832    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988836    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988839    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988842    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988845    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 22:08:41.443064 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988848    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988852    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988856    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988859    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988865    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988869    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988873    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988876    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988885    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988889    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988892    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988894    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988897    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988900    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988903    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988906    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988909    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988911    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988914    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988917    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 22:08:41.444407 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988920    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988922    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988926    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988928    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988932    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988934    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988937    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988940    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988943    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988945    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.988948    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.988954    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989073    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989077    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 22:08:41.445212 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989081    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989085    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989089    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989092    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989095    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989098    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989101    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989104    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989106    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989109    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989112    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989115    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989117    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989120    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989123    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989125    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989128    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989131    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989134    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989136    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 22:08:41.447391 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989139    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989142    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989145    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989147    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989150    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989153    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989156    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989159    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989162    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989165    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989168    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989171    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989175    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989178    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989182    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989185    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989188    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989191    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989194    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 22:08:41.448256 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989196    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989199    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989203    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989207    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989210    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989213    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989216    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989219    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989222    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989225    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989228    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989232    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989234    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989237    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989240    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989244    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989247    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989249    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989252    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989255    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 22:08:41.449573 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989257    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989260    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989263    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989266    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989269    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989273    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989276    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989280    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989284    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989286    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989291    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989294    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989297    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989300    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989322    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989327    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989332    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989335    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989338    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 22:08:41.450450 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989341    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989344    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989347    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989349    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989352    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989355    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989358    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989361    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989364    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989367    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989370    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989373    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989376    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989379    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989381    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989384    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989387    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989390    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 22:08:41.451748 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:40.989394    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.989400    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.989531    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.991814    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.997202    2566 server.go:1019] "Starting client certificate rotation"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.997324    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.997361    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:40.999741    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.001002    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.016226    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.018653    2566 log.go:25] "Validated CRI v1 image API"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.019526    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 22:08:41.453895 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.022126    2566 fs.go:135] Filesystem UUIDs: map[175b60b1-190d-4b31-9dda-e31d8499450a:/dev/nvme0n1p3 4f3fe9ec-faff-4b02-84b5-85e41e9af91e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 09 22:08:41.454270 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.022141    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.029426    2566 manager.go:216] Machine: {Timestamp:2026-07-09 22:08:41.027450927 +0000 UTC m=+0.918481436 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3136535 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec283e293d297434513064fcc8a45df1 SystemUUID:ec283e29-3d29-7434-5130-64fcc8a45df1 BootID:2d5e6af3-5e3f-46e4-9162-4af367625b6f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c4:25:b7:e5:47 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c4:25:b7:e5:47 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:9b:08:ac:19:12 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.029539    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.029642    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.032924    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.032952    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-248.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.033689    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.033700    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 22:08:41.454309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.033709    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.034406    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.035699    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.038796    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.038916    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.039641    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.039656    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.039672    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.039683    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.039696    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.040128    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 22:08:41.454582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.040149    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 22:08:41.506946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.506917    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 09 22:08:41.508984 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.508963    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 22:08:41.509087 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.509005    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 22:08:41.510658 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510641    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510672    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510682    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510694    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510707    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510716    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510736    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510745    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 22:08:41.510752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510756    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 22:08:41.511043 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510766    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 22:08:41.511043 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510806    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 22:08:41.511043 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510807    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 22:08:41.511043 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510840    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rx5xs"
Jul 09 22:08:41.511043 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.510817    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 22:08:41.511567 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.511556    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 22:08:41.511619 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.511572    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 22:08:41.516253 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.516237    2566 server.go:1268] "Started kubelet"
Jul 09 22:08:41.516354 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.516310    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 22:08:41.516409 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.516377    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 22:08:41.517222 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.517188    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 22:08:41.517348 ip-10-0-141-248 systemd[1]: Started Kubernetes Kubelet.
Jul 09 22:08:41.518114 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.518097    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 22:08:41.518646 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.518626    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 22:08:41.520236 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.520184    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-248.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 22:08:41.520236 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.520193    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 22:08:41.520459 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.520434    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 22:08:41.522361 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.522339    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 22:08:41.524802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.524782    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 22:08:41.524802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.524790    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 22:08:41.524943 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.524911    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525422    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525439    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525465    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525599    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525608    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525750    2566 factory.go:55] Registering systemd factory
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525764    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525983    2566 factory.go:153] Registering CRI-O factory
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.525998    2566 factory.go:223] Registration of the crio container factory successfully
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.526055    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.526081    2566 factory.go:103] Registering Raw factory
Jul 09 22:08:41.526204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.526099    2566 manager.go:1196] Started watching for new ooms in manager
Jul 09 22:08:41.526914 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.526825    2566 manager.go:349] Starting recovery of all containers
Jul 09 22:08:41.529994 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.529961    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 22:08:41.536177 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.536118    2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-248.ec2.internal\" not found" node="ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.539094 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.539062    2566 manager.go:354] Recovery completed
Jul 09 22:08:41.543846 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.543833    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.549284 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549267    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.549366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549302    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.549366 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549316    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.549819 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549802    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 22:08:41.549819 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549818    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 22:08:41.549819 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549826    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.549977 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.549839    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 22:08:41.550633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.550607    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.550633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.550635    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.550745 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.550648    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.551183 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.551171    2566 policy_none.go:49] "None policy: Start"
Jul 09 22:08:41.551227 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.551187    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 22:08:41.551227 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.551198    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 22:08:41.552314 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.552296    2566 policy_none.go:47] "Start"
Jul 09 22:08:41.590850 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.590836    2566 manager.go:337] "Starting Device Plugin manager"
Jul 09 22:08:41.590921 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.590871    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 22:08:41.590921 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.590885    2566 server.go:87] "Starting device plugin registration server"
Jul 09 22:08:41.591175 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.591158    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 22:08:41.591279 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.591176    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 22:08:41.591404 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.591284    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 22:08:41.591404 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.591366    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 22:08:41.591404 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.591373    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 22:08:41.591994 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.591968    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 22:08:41.592066 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.592032    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:41.671586 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.671553    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 22:08:41.672855 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.672838    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 22:08:41.672953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.672894    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 22:08:41.672953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.672925    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 22:08:41.673052 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.672973    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 22:08:41.676152 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.676134    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 22:08:41.692033 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.692020    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.692898 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.692879    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.692994 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.692913    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.692994 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.692929    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.692994 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.692956    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.715147 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.715130    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.715193 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.715152    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-248.ec2.internal\": node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:41.774491 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.774457    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"]
Jul 09 22:08:41.774592 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.774537    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.775521 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.775504    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.775576 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.775535    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.775576 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.775546    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.777112 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.777099    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.777280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.777249    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.777334 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.777298    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.778793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.778778    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.778856 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.778796    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.778856 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.778809    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.778856 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.778821    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.778856 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.778833    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.778973 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.778823    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.780345 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.780313    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.780397 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.780364    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 22:08:41.781125 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.781110    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 22:08:41.781187 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.781136    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 22:08:41.781187 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.781153    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 22:08:41.804222 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.804203    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-248.ec2.internal\" not found" node="ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.808991 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.808976    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-248.ec2.internal\" not found" node="ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.823967 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.823950    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:41.827903 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.827887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75c20c336a43f381c5b679d63baebd8c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.ec2.internal\" (UID: \"75c20c336a43f381c5b679d63baebd8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.827958 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.827912    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb91b37e3422fea39898c6fbc7bd0206-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal\" (UID: \"eb91b37e3422fea39898c6fbc7bd0206\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.827958 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.827932    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb91b37e3422fea39898c6fbc7bd0206-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal\" (UID: \"eb91b37e3422fea39898c6fbc7bd0206\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.924852 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:41.924765    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:41.928346 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.928330    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75c20c336a43f381c5b679d63baebd8c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.ec2.internal\" (UID: \"75c20c336a43f381c5b679d63baebd8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.928411 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.928356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb91b37e3422fea39898c6fbc7bd0206-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal\" (UID: \"eb91b37e3422fea39898c6fbc7bd0206\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.928411 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.928376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb91b37e3422fea39898c6fbc7bd0206-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal\" (UID: \"eb91b37e3422fea39898c6fbc7bd0206\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.928477 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.928428    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/75c20c336a43f381c5b679d63baebd8c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-248.ec2.internal\" (UID: \"75c20c336a43f381c5b679d63baebd8c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.928477 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.928467    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb91b37e3422fea39898c6fbc7bd0206-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal\" (UID: \"eb91b37e3422fea39898c6fbc7bd0206\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.928541 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.928427    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb91b37e3422fea39898c6fbc7bd0206-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal\" (UID: \"eb91b37e3422fea39898c6fbc7bd0206\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:41.993067 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.993024    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 22:08:41.993214 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.993197    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 22:08:41.993276 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:41.993230    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 22:08:42.026589 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.026561    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.107216 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.107169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:42.111617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.111600    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:42.127398 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.127370    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.227966 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.227882    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.328176 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.328120    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.428639 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.428588    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.519692 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.519587    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 22:03:41 +0000 UTC" deadline="2027-12-17 11:38:46.968538482 +0000 UTC"
Jul 09 22:08:42.519692 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.519630    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12613h30m4.448913118s"
Jul 09 22:08:42.529099 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.529071    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.629696 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:42.629661    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-248.ec2.internal\" not found"
Jul 09 22:08:42.641637 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.641610    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 22:08:42.679851 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:42.679817    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb91b37e3422fea39898c6fbc7bd0206.slice/crio-a13e281d08698c58aaf3b13c54615eac39a44cf33ad0ad706291aacef08a0e88 WatchSource:0}: Error finding container a13e281d08698c58aaf3b13c54615eac39a44cf33ad0ad706291aacef08a0e88: Status 404 returned error can't find the container with id a13e281d08698c58aaf3b13c54615eac39a44cf33ad0ad706291aacef08a0e88
Jul 09 22:08:42.680111 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:42.680097    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c20c336a43f381c5b679d63baebd8c.slice/crio-20979e9b02f1b6f3044dd0458a1009362ae233dc7f0068bc2e85432ad1dd6ffa WatchSource:0}: Error finding container 20979e9b02f1b6f3044dd0458a1009362ae233dc7f0068bc2e85432ad1dd6ffa: Status 404 returned error can't find the container with id 20979e9b02f1b6f3044dd0458a1009362ae233dc7f0068bc2e85432ad1dd6ffa
Jul 09 22:08:42.684448 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.684429    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 22:08:42.726529 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.726502    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:42.745275 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.745240    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 22:08:42.747603 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.747589    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal"
Jul 09 22:08:42.761741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.761722    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 22:08:42.825594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:42.825511    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 22:08:43.041455 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.041428    2566 apiserver.go:52] "Watching apiserver"
Jul 09 22:08:43.047166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.047144    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 22:08:43.047541 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.047519    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-mct98","openshift-multus/network-metrics-daemon-8xh4f","openshift-network-operator/iptables-alerter-wvvj7","kube-system/global-pull-secret-syncer-dx7b8","kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr","openshift-dns/node-resolver-2r5v5","openshift-multus/multus-additional-cni-plugins-8znkb","openshift-network-diagnostics/network-check-target-q87cw","openshift-ovn-kubernetes/ovnkube-node-r24c5","kube-system/konnectivity-agent-r76ff","openshift-cluster-node-tuning-operator/tuned-ftp5j","openshift-image-registry/node-ca-8z5xq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal"]
Jul 09 22:08:43.049834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.049817    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.051070 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.051047    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.051157 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.051133    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:43.052362 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.052345    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.054160 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.054134    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.054264 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.054208    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:43.055270 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.055251    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.055390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.055329    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 22:08:43.055524 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.055504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bw9k2\""
Jul 09 22:08:43.055524 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.055520    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.056390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.056368    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 22:08:43.056390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.056368    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.056688 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.056670    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.056791 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.056732    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.056854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.056733    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.057240 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.057220    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wsjtt\""
Jul 09 22:08:43.058197 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.058180    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.058634 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.058613    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.058801 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.058784    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 22:08:43.058877 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.058865    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.059033 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.059018    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-nw9v6\""
Jul 09 22:08:43.059110 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.059042    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 22:08:43.059169 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.059144    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.059226 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.059186    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.059344 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.059323    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-b9cfx\""
Jul 09 22:08:43.059546 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.059529    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:43.059618 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.059600    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:43.060804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.060787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 22:08:43.061013 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.060996    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-swtc5\""
Jul 09 22:08:43.061093 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.061045    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 22:08:43.061356 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.061327    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.063309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.063292    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.064863 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.064741    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.068166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.066894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.068166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.067881    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 22:08:43.068166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.067994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lbphc\""
Jul 09 22:08:43.068166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.068076    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 22:08:43.069750 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.069730    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.069832 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.069821    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 22:08:43.069875 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.069832    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 22:08:43.069875 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.069732    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 22:08:43.069959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.069732    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p7n2t\""
Jul 09 22:08:43.070001 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.069963    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.070353 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.070334    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 22:08:43.072846 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.072825    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.072941 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.072919    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-cc2kh\""
Jul 09 22:08:43.073079 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.073064    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 22:08:43.074095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.074080    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 22:08:43.074231 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.074209    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s8vdc\""
Jul 09 22:08:43.074231 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.074226    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.074799 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.074776    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 22:08:43.127280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.127260    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 22:08:43.134223 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.134315 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-cnibin\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.134315 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f01b8a21-8742-4b0e-8ef6-eeed7718e420-cni-binary-copy\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.134315 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134276    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-socket-dir-parent\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.134315 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134293    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-conf-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.134315 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134311    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/399349ba-0655-4074-b684-4e43c64ddfe3-kubelet-config\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.134315 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134327    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-ovnkube-script-lib\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134343    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxkr9\" (UniqueName: \"kubernetes.io/projected/786bc324-14ee-4525-8575-c2d218327c65-kube-api-access-hxkr9\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-systemd\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-tuned\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94f487f9-70be-4b90-a26c-4f235797dd0f-host\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134440    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysconfig\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134464    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysctl-d\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134487    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-cni-multus\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.134518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134505    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqcch\" (UniqueName: \"kubernetes.io/projected/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-kube-api-access-tqcch\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134522    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ff8432-417c-4f78-b181-4f44077cf60f-tmp-dir\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-run-netns\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134577    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-ovn\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134594    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e274313c-d93b-4c25-b548-448e893e974e-iptables-alerter-script\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87ff8432-417c-4f78-b181-4f44077cf60f-hosts-file\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rct5l\" (UniqueName: \"kubernetes.io/projected/87ff8432-417c-4f78-b181-4f44077cf60f-kube-api-access-rct5l\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-ovnkube-config\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysctl-conf\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q27c6\" (UniqueName: \"kubernetes.io/projected/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-kube-api-access-q27c6\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.134793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/399349ba-0655-4074-b684-4e43c64ddfe3-dbus\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134817    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59036915-3574-439b-bdd6-11bbde0bc1dc-konnectivity-ca\") pod \"konnectivity-agent-r76ff\" (UID: \"59036915-3574-439b-bdd6-11bbde0bc1dc\") " pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-run-ovn-kubernetes\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134881    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134909    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-env-overrides\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/94f487f9-70be-4b90-a26c-4f235797dd0f-serviceca\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134973    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-sys\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.134999    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-var-lib-kubelet\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135023    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-cni-bin\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135058    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-daemon-config\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135079    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-os-release\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6gr\" (UniqueName: \"kubernetes.io/projected/da8369e8-7d31-4ff5-a807-ba6165b0b90b-kube-api-access-pd6gr\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135123    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-slash\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-log-socket\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-cni-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135182    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135217    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-cni-bin\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-sys-fs\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135249    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-system-cni-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135297    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-k8s-cni-cncf-io\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjsjk\" (UniqueName: \"kubernetes.io/projected/e274313c-d93b-4c25-b548-448e893e974e-kube-api-access-gjsjk\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135340    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-node-log\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135383    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-kubernetes\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135413    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-host\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135433    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d97bx\" (UniqueName: \"kubernetes.io/projected/f01b8a21-8742-4b0e-8ef6-eeed7718e420-kube-api-access-d97bx\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-registration-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.135807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-device-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135503    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135531    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-cni-netd\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135554    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-run\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-netns\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135590    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-hostroot\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135616    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-os-release\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135693    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59036915-3574-439b-bdd6-11bbde0bc1dc-agent-certs\") pod \"konnectivity-agent-r76ff\" (UID: \"59036915-3574-439b-bdd6-11bbde0bc1dc\") " pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-var-lib-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm5f6\" (UniqueName: \"kubernetes.io/projected/94f487f9-70be-4b90-a26c-4f235797dd0f-kube-api-access-sm5f6\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e274313c-d93b-4c25-b548-448e893e974e-host-slash\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cnibin\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135828    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cni-binary-copy\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.136372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135853    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/786bc324-14ee-4525-8575-c2d218327c65-ovn-node-metrics-cert\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-modprobe-d\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135902    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-multus-certs\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-socket-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135953    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652p4\" (UniqueName: \"kubernetes.io/projected/96332960-3e27-4437-b375-7c16a4a8a2ea-kube-api-access-652p4\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.135978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-kubelet\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136003    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-systemd\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-etc-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136053    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136079    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-lib-modules\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136119    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-kubelet\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136153    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-tmp\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.136861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136194    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:43.137436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-systemd-units\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.137436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-etc-kubernetes\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.137436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.136271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-system-cni-dir\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237406 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-run\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237425    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-netns\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237449    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-hostroot\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237482    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-os-release\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237534    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237566    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-hostroot\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237587    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-os-release\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59036915-3574-439b-bdd6-11bbde0bc1dc-agent-certs\") pod \"konnectivity-agent-r76ff\" (UID: \"59036915-3574-439b-bdd6-11bbde0bc1dc\") " pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237536    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-run\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237633    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-var-lib-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237656    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-netns\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237670    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm5f6\" (UniqueName: \"kubernetes.io/projected/94f487f9-70be-4b90-a26c-4f235797dd0f-kube-api-access-sm5f6\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237681    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-var-lib-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237716    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e274313c-d93b-4c25-b548-448e893e974e-host-slash\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237750    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cnibin\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cni-binary-copy\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237819    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e274313c-d93b-4c25-b548-448e893e974e-host-slash\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/786bc324-14ee-4525-8575-c2d218327c65-ovn-node-metrics-cert\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237859    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cnibin\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-modprobe-d\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-multus-certs\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.237959 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.237968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-socket-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238002    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-modprobe-d\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652p4\" (UniqueName: \"kubernetes.io/projected/96332960-3e27-4437-b375-7c16a4a8a2ea-kube-api-access-652p4\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238030    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-multus-certs\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-kubelet\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-systemd\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238164    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-socket-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238176    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-kubelet\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238384    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-systemd\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238424    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238482    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-cni-binary-copy\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.238649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238163    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-etc-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238701    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-lib-modules\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-kubelet\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238775    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-tmp\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238778    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238774    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-etc-openvswitch\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238808    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238821    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-kubelet\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238862    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:43.239177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239050    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-lib-modules\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239351    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/da8369e8-7d31-4ff5-a807-ba6165b0b90b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.238889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-systemd-units\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-etc-kubernetes\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-system-cni-dir\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239468    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239495    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-cnibin\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239503    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-etc-kubernetes\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f01b8a21-8742-4b0e-8ef6-eeed7718e420-cni-binary-copy\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239540    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-socket-dir-parent\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239544    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-systemd-units\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-conf-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239638    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-conf-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239648    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-cnibin\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/399349ba-0655-4074-b684-4e43c64ddfe3-kubelet-config\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.239820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239698    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-system-cni-dir\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-ovnkube-script-lib\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxkr9\" (UniqueName: \"kubernetes.io/projected/786bc324-14ee-4525-8575-c2d218327c65-kube-api-access-hxkr9\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-systemd\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239797    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-tuned\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239821    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94f487f9-70be-4b90-a26c-4f235797dd0f-host\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysconfig\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239842    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da8369e8-7d31-4ff5-a807-ba6165b0b90b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysctl-d\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-cni-multus\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239954    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-cni-multus\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240011    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-systemd\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240137    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/399349ba-0655-4074-b684-4e43c64ddfe3-kubelet-config\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.239685    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-socket-dir-parent\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240351    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f01b8a21-8742-4b0e-8ef6-eeed7718e420-cni-binary-copy\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.240633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240450    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysconfig\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240501    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/94f487f9-70be-4b90-a26c-4f235797dd0f-host\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240599    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysctl-d\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240630    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqcch\" (UniqueName: \"kubernetes.io/projected/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-kube-api-access-tqcch\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240660    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ff8432-417c-4f78-b181-4f44077cf60f-tmp-dir\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240685    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-run-netns\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-ovn\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240733    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e274313c-d93b-4c25-b548-448e893e974e-iptables-alerter-script\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240757    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87ff8432-417c-4f78-b181-4f44077cf60f-hosts-file\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240804    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rct5l\" (UniqueName: \"kubernetes.io/projected/87ff8432-417c-4f78-b181-4f44077cf60f-kube-api-access-rct5l\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240813    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-ovnkube-script-lib\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-ovnkube-config\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240881    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-run-ovn\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.241463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysctl-conf\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240923    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q27c6\" (UniqueName: \"kubernetes.io/projected/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-kube-api-access-q27c6\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240943    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/399349ba-0655-4074-b684-4e43c64ddfe3-dbus\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59036915-3574-439b-bdd6-11bbde0bc1dc-konnectivity-ca\") pod \"konnectivity-agent-r76ff\" (UID: \"59036915-3574-439b-bdd6-11bbde0bc1dc\") " pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.240993    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-run-ovn-kubernetes\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-env-overrides\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/94f487f9-70be-4b90-a26c-4f235797dd0f-serviceca\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-sys\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241113    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-var-lib-kubelet\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241138    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-cni-bin\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-daemon-config\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-os-release\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241216    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6gr\" (UniqueName: \"kubernetes.io/projected/da8369e8-7d31-4ff5-a807-ba6165b0b90b-kube-api-access-pd6gr\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.242115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241243    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-slash\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-log-socket\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-cni-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241355    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-cni-bin\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241441    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-sys-fs\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241444    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-run-ovn-kubernetes\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-system-cni-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-k8s-cni-cncf-io\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjsjk\" (UniqueName: \"kubernetes.io/projected/e274313c-d93b-4c25-b548-448e893e974e-kube-api-access-gjsjk\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-node-log\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-kubernetes\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-host\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.242728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241606    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-sysctl-conf\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241629    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d97bx\" (UniqueName: \"kubernetes.io/projected/f01b8a21-8742-4b0e-8ef6-eeed7718e420-kube-api-access-d97bx\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-registration-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-device-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-cni-netd\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241810    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-cni-netd\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241858    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.241916    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e274313c-d93b-4c25-b548-448e893e974e-iptables-alerter-script\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.242011    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.242076    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:43.74205533 +0000 UTC m=+3.633085839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242150    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/399349ba-0655-4074-b684-4e43c64ddfe3-dbus\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242224    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-env-overrides\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242402    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-tmp\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.243410 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242572    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/87ff8432-417c-4f78-b181-4f44077cf60f-hosts-file\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242589    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87ff8432-417c-4f78-b181-4f44077cf60f-tmp-dir\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242654    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-run-netns\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242661    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/94f487f9-70be-4b90-a26c-4f235797dd0f-serviceca\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242703    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-node-log\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242732    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-kubernetes\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242743    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/786bc324-14ee-4525-8575-c2d218327c65-ovn-node-metrics-cert\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242790    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-host\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242802    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-system-cni-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242827    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-sys-fs\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242848    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-run-k8s-cni-cncf-io\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.242747    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-cni-bin\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243082    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-registration-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243088    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/786bc324-14ee-4525-8575-c2d218327c65-ovnkube-config\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243140    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-device-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.244158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243167    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-sys\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243229    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-var-lib-kubelet\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.243237    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243273    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-host-var-lib-cni-bin\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.243297    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:08:43.743280691 +0000 UTC m=+3.634311188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243223    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/59036915-3574-439b-bdd6-11bbde0bc1dc-agent-certs\") pod \"konnectivity-agent-r76ff\" (UID: \"59036915-3574-439b-bdd6-11bbde0bc1dc\") " pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243332    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-host-slash\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243400    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-cni-dir\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243401    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f01b8a21-8742-4b0e-8ef6-eeed7718e420-os-release\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243464    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243492    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/59036915-3574-439b-bdd6-11bbde0bc1dc-konnectivity-ca\") pod \"konnectivity-agent-r76ff\" (UID: \"59036915-3574-439b-bdd6-11bbde0bc1dc\") " pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243544    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243688    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-etc-tuned\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243710    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f01b8a21-8742-4b0e-8ef6-eeed7718e420-multus-daemon-config\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.244790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.243798    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/786bc324-14ee-4525-8575-c2d218327c65-log-socket\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.263548 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.263501    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqcch\" (UniqueName: \"kubernetes.io/projected/8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a-kube-api-access-tqcch\") pod \"aws-ebs-csi-driver-node-6tnjr\" (UID: \"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.263725 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.263710    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxkr9\" (UniqueName: \"kubernetes.io/projected/786bc324-14ee-4525-8575-c2d218327c65-kube-api-access-hxkr9\") pod \"ovnkube-node-r24c5\" (UID: \"786bc324-14ee-4525-8575-c2d218327c65\") " pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.264973 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.264946    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q27c6\" (UniqueName: \"kubernetes.io/projected/ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39-kube-api-access-q27c6\") pod \"tuned-ftp5j\" (UID: \"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39\") " pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.272626 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.272597    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm5f6\" (UniqueName: \"kubernetes.io/projected/94f487f9-70be-4b90-a26c-4f235797dd0f-kube-api-access-sm5f6\") pod \"node-ca-8z5xq\" (UID: \"94f487f9-70be-4b90-a26c-4f235797dd0f\") " pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.286882 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.285787    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:08:43.286882 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.285813    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:08:43.286882 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.285828    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:43.286882 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.285901    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:08:43.785879228 +0000 UTC m=+3.676909751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:43.286882 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.286502    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-652p4\" (UniqueName: \"kubernetes.io/projected/96332960-3e27-4437-b375-7c16a4a8a2ea-kube-api-access-652p4\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.287480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.287456    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d97bx\" (UniqueName: \"kubernetes.io/projected/f01b8a21-8742-4b0e-8ef6-eeed7718e420-kube-api-access-d97bx\") pod \"multus-mct98\" (UID: \"f01b8a21-8742-4b0e-8ef6-eeed7718e420\") " pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.287666 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.287635    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjsjk\" (UniqueName: \"kubernetes.io/projected/e274313c-d93b-4c25-b548-448e893e974e-kube-api-access-gjsjk\") pod \"iptables-alerter-wvvj7\" (UID: \"e274313c-d93b-4c25-b548-448e893e974e\") " pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.289259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.289226    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6gr\" (UniqueName: \"kubernetes.io/projected/da8369e8-7d31-4ff5-a807-ba6165b0b90b-kube-api-access-pd6gr\") pod \"multus-additional-cni-plugins-8znkb\" (UID: \"da8369e8-7d31-4ff5-a807-ba6165b0b90b\") " pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.289428 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.289411    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rct5l\" (UniqueName: \"kubernetes.io/projected/87ff8432-417c-4f78-b181-4f44077cf60f-kube-api-access-rct5l\") pod \"node-resolver-2r5v5\" (UID: \"87ff8432-417c-4f78-b181-4f44077cf60f\") " pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.342820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.342737    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 22:08:43.360809 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.360782    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mct98"
Jul 09 22:08:43.371493 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.371466    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wvvj7"
Jul 09 22:08:43.373808 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.373744    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf01b8a21_8742_4b0e_8ef6_eeed7718e420.slice/crio-126b6d136cffe732f566f0928edf1da493fed5c9e8e542c844c4a68df0c4276b WatchSource:0}: Error finding container 126b6d136cffe732f566f0928edf1da493fed5c9e8e542c844c4a68df0c4276b: Status 404 returned error can't find the container with id 126b6d136cffe732f566f0928edf1da493fed5c9e8e542c844c4a68df0c4276b
Jul 09 22:08:43.379272 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.379227    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr"
Jul 09 22:08:43.380184 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.380113    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode274313c_d93b_4c25_b548_448e893e974e.slice/crio-b743ae449628089554bbbc1f4f4a00b8a679c141f6d02fa1bb98870b15cd0e37 WatchSource:0}: Error finding container b743ae449628089554bbbc1f4f4a00b8a679c141f6d02fa1bb98870b15cd0e37: Status 404 returned error can't find the container with id b743ae449628089554bbbc1f4f4a00b8a679c141f6d02fa1bb98870b15cd0e37
Jul 09 22:08:43.382417 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.382398    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2r5v5"
Jul 09 22:08:43.388135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.388113    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8znkb"
Jul 09 22:08:43.389746 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.389720    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bdbb179_0f4c_4c47_8e96_a6f7359bbb2a.slice/crio-f7f3ce26eb1aa9311f7d6c3727d71c27da28437a74993091f6fec90c02155573 WatchSource:0}: Error finding container f7f3ce26eb1aa9311f7d6c3727d71c27da28437a74993091f6fec90c02155573: Status 404 returned error can't find the container with id f7f3ce26eb1aa9311f7d6c3727d71c27da28437a74993091f6fec90c02155573
Jul 09 22:08:43.392166 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.392133    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ff8432_417c_4f78_b181_4f44077cf60f.slice/crio-483b599b3617afb5199d53b40d9503fee85ad5bd50932020ed028d17262009ab WatchSource:0}: Error finding container 483b599b3617afb5199d53b40d9503fee85ad5bd50932020ed028d17262009ab: Status 404 returned error can't find the container with id 483b599b3617afb5199d53b40d9503fee85ad5bd50932020ed028d17262009ab
Jul 09 22:08:43.393931 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.393912    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:08:43.397940 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.397910    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8369e8_7d31_4ff5_a807_ba6165b0b90b.slice/crio-d169f5d9a0efb8092ddd83ba36eec89c4afdba49e3a6298997fbbd2596a3670d WatchSource:0}: Error finding container d169f5d9a0efb8092ddd83ba36eec89c4afdba49e3a6298997fbbd2596a3670d: Status 404 returned error can't find the container with id d169f5d9a0efb8092ddd83ba36eec89c4afdba49e3a6298997fbbd2596a3670d
Jul 09 22:08:43.399854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.399809    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:08:43.403833 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.403813    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786bc324_14ee_4525_8575_c2d218327c65.slice/crio-b25166d7e803103ac41167a4c328b571ff08e0657a95d4b32bba2cc7aaaaf7a8 WatchSource:0}: Error finding container b25166d7e803103ac41167a4c328b571ff08e0657a95d4b32bba2cc7aaaaf7a8: Status 404 returned error can't find the container with id b25166d7e803103ac41167a4c328b571ff08e0657a95d4b32bba2cc7aaaaf7a8
Jul 09 22:08:43.405670 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.405649    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ftp5j"
Jul 09 22:08:43.410437 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.410412    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59036915_3574_439b_bdd6_11bbde0bc1dc.slice/crio-9de30bb980a43b93574b074f6bbdbffe334519a086e7f271d2b8b1b75c9ca25f WatchSource:0}: Error finding container 9de30bb980a43b93574b074f6bbdbffe334519a086e7f271d2b8b1b75c9ca25f: Status 404 returned error can't find the container with id 9de30bb980a43b93574b074f6bbdbffe334519a086e7f271d2b8b1b75c9ca25f
Jul 09 22:08:43.410537 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.410482    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8z5xq"
Jul 09 22:08:43.414704 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.414678    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7ccbf1_edd0_4fe8_bdec_93079a8ffe39.slice/crio-0796c481e60a54540c4e001d2612e0fcdce521d62f3359fc918269872090726e WatchSource:0}: Error finding container 0796c481e60a54540c4e001d2612e0fcdce521d62f3359fc918269872090726e: Status 404 returned error can't find the container with id 0796c481e60a54540c4e001d2612e0fcdce521d62f3359fc918269872090726e
Jul 09 22:08:43.422403 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:08:43.422383    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94f487f9_70be_4b90_a26c_4f235797dd0f.slice/crio-5d0430cb114acbe6895a797f8552d52cfba68500aea73fb33c50365e4ce048dd WatchSource:0}: Error finding container 5d0430cb114acbe6895a797f8552d52cfba68500aea73fb33c50365e4ce048dd: Status 404 returned error can't find the container with id 5d0430cb114acbe6895a797f8552d52cfba68500aea73fb33c50365e4ce048dd
Jul 09 22:08:43.475216 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.475173    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 22:08:43.526596 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.526561    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 22:08:43.546848 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.546792    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 22:08:43.602604 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.602527    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zsc7n"
Jul 09 22:08:43.616807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.616776    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zsc7n"
Jul 09 22:08:43.679760 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.679656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r76ff" event={"ID":"59036915-3574-439b-bdd6-11bbde0bc1dc","Type":"ContainerStarted","Data":"9de30bb980a43b93574b074f6bbdbffe334519a086e7f271d2b8b1b75c9ca25f"}
Jul 09 22:08:43.681879 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.681799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerStarted","Data":"d169f5d9a0efb8092ddd83ba36eec89c4afdba49e3a6298997fbbd2596a3670d"}
Jul 09 22:08:43.683448 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.683402    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mct98" event={"ID":"f01b8a21-8742-4b0e-8ef6-eeed7718e420","Type":"ContainerStarted","Data":"126b6d136cffe732f566f0928edf1da493fed5c9e8e542c844c4a68df0c4276b"}
Jul 09 22:08:43.685405 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.685372    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal" event={"ID":"75c20c336a43f381c5b679d63baebd8c","Type":"ContainerStarted","Data":"20979e9b02f1b6f3044dd0458a1009362ae233dc7f0068bc2e85432ad1dd6ffa"}
Jul 09 22:08:43.687967 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.687938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8z5xq" event={"ID":"94f487f9-70be-4b90-a26c-4f235797dd0f","Type":"ContainerStarted","Data":"5d0430cb114acbe6895a797f8552d52cfba68500aea73fb33c50365e4ce048dd"}
Jul 09 22:08:43.689740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.689715    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"b25166d7e803103ac41167a4c328b571ff08e0657a95d4b32bba2cc7aaaaf7a8"}
Jul 09 22:08:43.691969 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.691929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2r5v5" event={"ID":"87ff8432-417c-4f78-b181-4f44077cf60f","Type":"ContainerStarted","Data":"483b599b3617afb5199d53b40d9503fee85ad5bd50932020ed028d17262009ab"}
Jul 09 22:08:43.701026 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.698518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr" event={"ID":"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a","Type":"ContainerStarted","Data":"f7f3ce26eb1aa9311f7d6c3727d71c27da28437a74993091f6fec90c02155573"}
Jul 09 22:08:43.704371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.703840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wvvj7" event={"ID":"e274313c-d93b-4c25-b548-448e893e974e","Type":"ContainerStarted","Data":"b743ae449628089554bbbc1f4f4a00b8a679c141f6d02fa1bb98870b15cd0e37"}
Jul 09 22:08:43.705761 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.705248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal" event={"ID":"eb91b37e3422fea39898c6fbc7bd0206","Type":"ContainerStarted","Data":"a13e281d08698c58aaf3b13c54615eac39a44cf33ad0ad706291aacef08a0e88"}
Jul 09 22:08:43.707431 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.707382    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ftp5j" event={"ID":"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39","Type":"ContainerStarted","Data":"0796c481e60a54540c4e001d2612e0fcdce521d62f3359fc918269872090726e"}
Jul 09 22:08:43.745386 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.745351    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:43.745538 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.745426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:43.745538 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.745534    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:43.745646 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.745596    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:08:44.745578607 +0000 UTC m=+4.636609100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:43.745713 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.745658    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:43.745713 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.745682    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:44.745675279 +0000 UTC m=+4.636705773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:43.846339 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:43.846265    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:43.846533 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.846478    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:08:43.846533 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.846503    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:08:43.846533 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.846514    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:43.846659 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:43.846577    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:08:44.846561352 +0000 UTC m=+4.737591850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:44.619892 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.619786    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 22:03:43 +0000 UTC" deadline="2028-02-21 08:09:50.459062116 +0000 UTC"
Jul 09 22:08:44.619892 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.619829    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14194h1m5.839238145s"
Jul 09 22:08:44.673804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.673179    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:44.673804 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.673340    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:44.673804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.673622    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:44.673804 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.673725    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:44.673804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.673734    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:44.674212 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.673884    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:44.754953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.754762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:44.754953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.754854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:44.755198 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.755084    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:44.755198 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.755159    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:08:46.755138072 +0000 UTC m=+6.646168574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:44.755619 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.755599    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:44.755719 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.755659    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:46.755642093 +0000 UTC m=+6.646672593 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:44.855893 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:44.855850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:44.856094 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.856058    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:08:44.856094 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.856081    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:08:44.856094 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.856096    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:44.856259 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:44.856159    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:08:46.856138905 +0000 UTC m=+6.747169401 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:45.620816 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:45.620769    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 22:03:43 +0000 UTC" deadline="2028-03-01 12:17:50.563129073 +0000 UTC"
Jul 09 22:08:45.620816 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:45.620817    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14414h9m4.942318471s"
Jul 09 22:08:46.674028 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:46.673996    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:46.674396 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.674159    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:46.674530 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:46.674511    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:46.674622 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.674602    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:46.674669 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:46.674650    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:46.674738 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.674712    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:46.775633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:46.775595    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:46.775885 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:46.775685    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:46.775885 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.775832    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:46.776016 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.775901    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:50.775882194 +0000 UTC m=+10.666912686 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:46.776355 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.776336    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:46.776439 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.776393    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:08:50.776378162 +0000 UTC m=+10.667408660 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:46.876519 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:46.876481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:46.876701 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.876631    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:08:46.876701 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.876653    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:08:46.876701 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.876666    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:46.876865 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:46.876726    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:08:50.876707666 +0000 UTC m=+10.767738174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:48.674223 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:48.673971    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:48.674673 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:48.674369    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:48.674673 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:48.674155    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:48.674673 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:48.674467    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:48.674673 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:48.674108    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:48.674673 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:48.674541    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:50.673617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:50.673575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:50.673617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:50.673604    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:50.674167 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:50.673575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:50.674167 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.673715    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:50.674167 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.673804    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:50.674167 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.673878    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:50.807792 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:50.807754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:50.807979 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:50.807812    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:50.807979 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.807920    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:50.807979 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.807942    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:50.808138 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.807992    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:08:58.807971676 +0000 UTC m=+18.699002174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:50.808138 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.808013    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:08:58.808002635 +0000 UTC m=+18.699033133 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:50.908363 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:50.908332    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:50.908560 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.908475    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:08:50.908560 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.908493    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:08:50.908560 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.908502    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:50.908715 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:50.908562    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:08:58.908543236 +0000 UTC m=+18.799573744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:52.674451 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:52.673977    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:52.674451 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:52.674012    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:52.674451 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:52.674035    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:52.674451 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:52.674157    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:52.675077 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:52.674614    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:52.675077 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:52.674695    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:54.673450 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:54.673405    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:54.673902 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:54.673468    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:54.673902 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:54.673480    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:54.673902 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:54.673582    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:54.673902 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:54.673666    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:54.673902 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:54.673763    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:56.674099 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:56.674062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:56.674538 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:56.674062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:56.674538 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:56.674185    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:56.674538 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:56.674310    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:56.674538 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:56.674062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:56.674538 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:56.674438    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:58.674019 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:58.673929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:58.674019 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:58.673967    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:58.674019 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:58.673965    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:58.674509 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.674050    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:08:58.674509 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.674147    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:08:58.674509 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.674197    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:08:58.869925 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:58.869885    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:08:58.870096 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:58.869950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:08:58.870096 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.870029    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:58.870096 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.870046    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:58.870265 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.870111    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:09:14.870088765 +0000 UTC m=+34.761119264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:08:58.870265 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.870132    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:09:14.870122135 +0000 UTC m=+34.761152628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:08:58.970866 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:08:58.970765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:08:58.971043 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.970926    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:08:58.971043 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.970951    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:08:58.971043 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.970963    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:08:58.971043 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:08:58.971029    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:09:14.971009936 +0000 UTC m=+34.862040440 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:09:00.673450 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:00.673410    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:00.673450 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:00.673433    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:00.673922 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:00.673445    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:00.673922 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:00.673539    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:00.673922 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:00.673605    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:00.673922 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:00.673702    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:02.673394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:02.673350    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:02.673394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:02.673373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:02.673394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:02.673350    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:02.673930 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:02.673486    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:02.673930 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:02.673581    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:02.673930 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:02.673682    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:03.752519 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.752282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ftp5j" event={"ID":"ee7ccbf1-edd0-4fe8-bdec-93079a8ffe39","Type":"ContainerStarted","Data":"8c7b711f7580af6685d5bc71a4a7ddfb0a3a77776bb4ac378f0f69c923908db4"}
Jul 09 22:09:03.754485 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.754460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mct98" event={"ID":"f01b8a21-8742-4b0e-8ef6-eeed7718e420","Type":"ContainerStarted","Data":"adc17231fe7134afe5470c68f6f451a2cb5967018ca685c1fd085e293b97a3f0"}
Jul 09 22:09:03.756047 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.756024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal" event={"ID":"75c20c336a43f381c5b679d63baebd8c","Type":"ContainerStarted","Data":"7acb6fe81c208a6a984fc8e24602f19ad06f40ff68ed4685cd41de339509232b"}
Jul 09 22:09:03.758130 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.758108    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"9dd8c51924f25df20590f5361e32895140ddecc418b481972ae37def6bd9a80f"}
Jul 09 22:09:03.758225 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.758164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"9a78355c5cfa8b172e83132b6513e4044a0674ce4d8a5241cd911a0320c5a8a3"}
Jul 09 22:09:03.839397 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.839349    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ftp5j" podStartSLOduration=2.323125247 podStartE2EDuration="21.839333967s" totalImagesPullingTime="19.51620872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:03.813376284 +0000 UTC m=+23.704406798" watchObservedRunningTime="2026-07-09 22:09:03.839333967 +0000 UTC m=+23.730364481"
Jul 09 22:09:03.839731 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.839708    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-248.ec2.internal" podStartSLOduration=21.839702327 podStartE2EDuration="21.839702327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:03.839500335 +0000 UTC m=+23.730530861" watchObservedRunningTime="2026-07-09 22:09:03.839702327 +0000 UTC m=+23.730732841"
Jul 09 22:09:03.872800 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:03.872712    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mct98" podStartSLOduration=2.281922125 podStartE2EDuration="21.872695338s" totalImagesPullingTime="19.590773213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:03.87130258 +0000 UTC m=+23.762333094" watchObservedRunningTime="2026-07-09 22:09:03.872695338 +0000 UTC m=+23.763725852"
Jul 09 22:09:04.674334 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.674099    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:04.674485 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.674151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:04.674485 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:04.674378    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:04.674485 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.674151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:04.674485 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:04.674423    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:04.674676 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:04.674496    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:04.761419 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.761385    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8z5xq" event={"ID":"94f487f9-70be-4b90-a26c-4f235797dd0f","Type":"ContainerStarted","Data":"93865803f7c9f71d68d4a084ad1eb31794a1fe91f96c366867e4fac949e07908"}
Jul 09 22:09:04.764350 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.764317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"ee01a523ad5df78be0dfaf7da7ba2c056a5dc69838cb3ee3692c2e41ac9ac374"}
Jul 09 22:09:04.764481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.764378    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"51535cfe234d645b8251e3fea2e03a9878277fa8c138251380488d09e555619d"}
Jul 09 22:09:04.764481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.764424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"2451bcc31c8bdc981ac5182e2b939469a8ad70cbbe14a66f9d238c8a3e0e12eb"}
Jul 09 22:09:04.764481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.764453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"f536768a40553bd3d5dda4579368034124d13a583faef6194618b8bbf0e9dc76"}
Jul 09 22:09:04.765737 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.765706    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2r5v5" event={"ID":"87ff8432-417c-4f78-b181-4f44077cf60f","Type":"ContainerStarted","Data":"d49cdef7e2847ee4157387ffa3aa9c6e3771c9e10ed8cf855db86e5c01c60dfa"}
Jul 09 22:09:04.767066 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.767045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr" event={"ID":"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a","Type":"ContainerStarted","Data":"502db1e6271d8e00dd2001ac7882d35f16b4644c42bf2fb08f7869570e59fc01"}
Jul 09 22:09:04.768383 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.768360    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wvvj7" event={"ID":"e274313c-d93b-4c25-b548-448e893e974e","Type":"ContainerStarted","Data":"565418094c3f917c291430139905140b0f46c76173d611213d21c06bc0a3939f"}
Jul 09 22:09:04.769721 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.769703    2566 generic.go:356] "Generic (PLEG): container finished" podID="eb91b37e3422fea39898c6fbc7bd0206" containerID="11e6bb249629676ea89400c6cd663636701668c297b20907158194f79f08d9e7" exitCode=0
Jul 09 22:09:04.769798 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.769762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal" event={"ID":"eb91b37e3422fea39898c6fbc7bd0206","Type":"ContainerDied","Data":"11e6bb249629676ea89400c6cd663636701668c297b20907158194f79f08d9e7"}
Jul 09 22:09:04.771079 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.771052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-r76ff" event={"ID":"59036915-3574-439b-bdd6-11bbde0bc1dc","Type":"ContainerStarted","Data":"a650c8cc0d03e357805700660cbac538123b2ec176ad6a695eab016b941d92af"}
Jul 09 22:09:04.772532 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.772510    2566 generic.go:356] "Generic (PLEG): container finished" podID="da8369e8-7d31-4ff5-a807-ba6165b0b90b" containerID="fef0cddbf310429b2ef61b54418273e0961235273871370da16e60574993c2cd" exitCode=0
Jul 09 22:09:04.772615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.772600    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerDied","Data":"fef0cddbf310429b2ef61b54418273e0961235273871370da16e60574993c2cd"}
Jul 09 22:09:04.778142 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.778094    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8z5xq" podStartSLOduration=3.316115521 podStartE2EDuration="22.77807745s" totalImagesPullingTime="19.461961929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:04.777739718 +0000 UTC m=+24.668770232" watchObservedRunningTime="2026-07-09 22:09:04.77807745 +0000 UTC m=+24.669107965"
Jul 09 22:09:04.796134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.796101    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wvvj7" podStartSLOduration=7.856051449 podStartE2EDuration="22.796088631s" totalImagesPullingTime="14.940037182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:04.796060325 +0000 UTC m=+24.687090838" watchObservedRunningTime="2026-07-09 22:09:04.796088631 +0000 UTC m=+24.687119144"
Jul 09 22:09:04.835815 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.835778    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-r76ff" podStartSLOduration=3.3154110709999998 podStartE2EDuration="22.835766146s" totalImagesPullingTime="19.520355075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:04.814387814 +0000 UTC m=+24.705418333" watchObservedRunningTime="2026-07-09 22:09:04.835766146 +0000 UTC m=+24.726796658"
Jul 09 22:09:04.836376 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:04.836341    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2r5v5" podStartSLOduration=3.297907174 podStartE2EDuration="22.836331852s" totalImagesPullingTime="19.538424678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:04.835608653 +0000 UTC m=+24.726639170" watchObservedRunningTime="2026-07-09 22:09:04.836331852 +0000 UTC m=+24.727362367"
Jul 09 22:09:05.634918 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.634891    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 22:09:05.748556 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.748532    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:09:05.749280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.749257    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:09:05.776812 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.776736    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr" event={"ID":"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a","Type":"ContainerStarted","Data":"9e7484901013532fb4fd7f249cb50f39cc6896b435a066176a6637e0f84add94"}
Jul 09 22:09:05.778904 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.778877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal" event={"ID":"eb91b37e3422fea39898c6fbc7bd0206","Type":"ContainerStarted","Data":"82772b4ca020423fc01532e16143bc51ea2f6618f8f6d9bd9845e8e50436eef2"}
Jul 09 22:09:05.779636 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.779619    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:09:05.779739 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.779690    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-r76ff"
Jul 09 22:09:05.800947 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:05.800905    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-248.ec2.internal" podStartSLOduration=23.800888276 podStartE2EDuration="23.800888276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:05.800543052 +0000 UTC m=+25.691573565" watchObservedRunningTime="2026-07-09 22:09:05.800888276 +0000 UTC m=+25.691918792"
Jul 09 22:09:06.609572 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.609476    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T22:09:05.634914929Z","UUID":"670084cf-d140-430c-bf31-ddb447268b18","Handler":null,"Name":"","Endpoint":""}
Jul 09 22:09:06.612390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.612364    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 22:09:06.612529 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.612408    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 22:09:06.673683 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.673654    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:06.673845 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.673664    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:06.673845 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:06.673771    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:06.673845 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.673811    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:06.674014 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:06.673900    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:06.674014 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:06.673981    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:06.784305 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:06.784274    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"f0dd89f5021b388dbca5a2e066d4695cc997b440023238c3a9df733683b82e3c"}
Jul 09 22:09:07.788765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:07.788533    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr" event={"ID":"8bdbb179-0f4c-4c47-8e96-a6f7359bbb2a","Type":"ContainerStarted","Data":"d31568d875481a2626f190aff34510e96ca6bd07f56dea1436507f6636b0137c"}
Jul 09 22:09:07.811185 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:07.811128    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6tnjr" podStartSLOduration=3.385263792 podStartE2EDuration="25.81110951s" totalImagesPullingTime="22.425845718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:07.810889026 +0000 UTC m=+27.701919552" watchObservedRunningTime="2026-07-09 22:09:07.81110951 +0000 UTC m=+27.702140026"
Jul 09 22:09:08.673167 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:08.673091    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:08.673311 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:08.673095    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:08.673311 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:08.673192    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:08.673311 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:08.673302    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:08.673429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:08.673102    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:08.673429 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:08.673374    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:08.793723 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:08.793684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" event={"ID":"786bc324-14ee-4525-8575-c2d218327c65","Type":"ContainerStarted","Data":"90e558d25d127a273ab93ef286ab46849000503a5ab61e99c3f8ffdd44ad2ada"}
Jul 09 22:09:08.834148 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:08.834095    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5" podStartSLOduration=6.781126997 podStartE2EDuration="26.834076013s" totalImagesPullingTime="20.052949016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:08.832446836 +0000 UTC m=+28.723477350" watchObservedRunningTime="2026-07-09 22:09:08.834076013 +0000 UTC m=+28.725106530"
Jul 09 22:09:09.797202 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:09.796958    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:09:09.797657 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:09.797281    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:09:09.797657 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:09.797336    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:09:09.812852 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:09.812826    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:09:09.813333 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:09.813310    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:09:10.539112 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:10.539019    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q87cw"]
Jul 09 22:09:10.539323 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:10.539152    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:10.539323 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:10.539271    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:10.539776 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:10.539752    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dx7b8"]
Jul 09 22:09:10.539904 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:10.539849    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:10.539960 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:10.539917    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:10.545992 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:10.545970    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8xh4f"]
Jul 09 22:09:10.546096 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:10.546083    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:10.546220 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:10.546182    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:11.628060 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:11.628023    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda8369e8_7d31_4ff5_a807_ba6165b0b90b.slice/crio-conmon-a05617dacf3463cdc7516bc0d7693e554693f9c8dcc103b701adaac299b40f47.scope\": RecentStats: unable to find data in memory cache]"
Jul 09 22:09:11.803285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:11.803194    2566 generic.go:356] "Generic (PLEG): container finished" podID="da8369e8-7d31-4ff5-a807-ba6165b0b90b" containerID="a05617dacf3463cdc7516bc0d7693e554693f9c8dcc103b701adaac299b40f47" exitCode=0
Jul 09 22:09:11.803407 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:11.803278    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerDied","Data":"a05617dacf3463cdc7516bc0d7693e554693f9c8dcc103b701adaac299b40f47"}
Jul 09 22:09:12.673842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:12.673770    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:12.673842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:12.673814    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:12.674359 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:12.673776    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:12.674359 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:12.673897    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:12.674359 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:12.674020    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:12.674359 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:12.674118    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:13.808298 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:13.808269    2566 generic.go:356] "Generic (PLEG): container finished" podID="da8369e8-7d31-4ff5-a807-ba6165b0b90b" containerID="0fd99b4b072709450f3306c6f3daa90147e72c9d2f30f05b9584bfc6525dfe7f" exitCode=0
Jul 09 22:09:13.808673 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:13.808330    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerDied","Data":"0fd99b4b072709450f3306c6f3daa90147e72c9d2f30f05b9584bfc6525dfe7f"}
Jul 09 22:09:14.674309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:14.674101    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:14.674460 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:14.674162    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:14.674460 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.674407    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dx7b8" podUID="399349ba-0655-4074-b684-4e43c64ddfe3"
Jul 09 22:09:14.674577 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:14.674162    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:14.674577 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.674532    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:09:14.674577 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.674538    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q87cw" podUID="2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f"
Jul 09 22:09:14.890698 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:14.890620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:14.890698 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:14.890665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:14.891055 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.890763    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:09:14.891055 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.890767    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 22:09:14.891055 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.890811    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:09:46.890798702 +0000 UTC m=+66.781829195 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 22:09:14.891055 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.890824    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret podName:399349ba-0655-4074-b684-4e43c64ddfe3 nodeName:}" failed. No retries permitted until 2026-07-09 22:09:46.89081804 +0000 UTC m=+66.781848532 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret") pod "global-pull-secret-syncer-dx7b8" (UID: "399349ba-0655-4074-b684-4e43c64ddfe3") : object "kube-system"/"original-pull-secret" not registered
Jul 09 22:09:14.991811 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:14.991776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:14.991963 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.991904    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 22:09:14.991963 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.991920    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 22:09:14.991963 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.991929    2566 projected.go:196] Error preparing data for projected volume kube-api-access-v6xl7 for pod openshift-network-diagnostics/network-check-target-q87cw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:09:14.992074 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:14.991974    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7 podName:2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f nodeName:}" failed. No retries permitted until 2026-07-09 22:09:46.991961843 +0000 UTC m=+66.882992335 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v6xl7" (UniqueName: "kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7") pod "network-check-target-q87cw" (UID: "2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 22:09:15.814712 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:15.814679    2566 generic.go:356] "Generic (PLEG): container finished" podID="da8369e8-7d31-4ff5-a807-ba6165b0b90b" containerID="9fc60492e3bfad7d319423ddfa961be233863d2f284746c407d5be7dbe062960" exitCode=0
Jul 09 22:09:15.814870 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:15.814738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerDied","Data":"9fc60492e3bfad7d319423ddfa961be233863d2f284746c407d5be7dbe062960"}
Jul 09 22:09:15.927391 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:15.927373    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-248.ec2.internal" event="NodeReady"
Jul 09 22:09:15.927681 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:15.927484    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 22:09:15.982551 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:15.982525    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cc865548f-9rtph"]
Jul 09 22:09:16.004744 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.004721    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.007331 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.007311    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 22:09:16.007331 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.007322    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 22:09:16.007557 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.007543    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 22:09:16.007933 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.007911    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jg4kh\""
Jul 09 22:09:16.011588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.011566    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cc865548f-9rtph"]
Jul 09 22:09:16.020888 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.020622    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 22:09:16.042541 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.042516    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g9qfh"]
Jul 09 22:09:16.060283 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.060235    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2xl48"]
Jul 09 22:09:16.060392 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.060373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.066799 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.066781    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 22:09:16.067370 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.067349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 22:09:16.067455 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.067423    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77st2\""
Jul 09 22:09:16.086409 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.086385    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xl48"]
Jul 09 22:09:16.086508 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.086475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.094573 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.094551    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rkrl7\""
Jul 09 22:09:16.094831 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.094811    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 22:09:16.094923 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.094866    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 22:09:16.095128 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.095110    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 22:09:16.095598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.095581    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g9qfh"]
Jul 09 22:09:16.101241 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-certificates\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101344 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-bound-sa-token\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101344 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pxp9\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-kube-api-access-5pxp9\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101344 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101344 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-ca-trust-extracted\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101484 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-trusted-ca\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101484 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-image-registry-private-configuration\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.101484 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.101457    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-installation-pull-secrets\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.202635 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.202635 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202638    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5478bd7-3b16-49c6-a902-e016f8f4e82d-config-volume\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-certificates\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202715    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-bound-sa-token\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pxp9\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-kube-api-access-5pxp9\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202756    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svmwf\" (UniqueName: \"kubernetes.io/projected/c5478bd7-3b16-49c6-a902-e016f8f4e82d-kube-api-access-svmwf\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202780    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4f8c700-e1c0-4abc-bda7-cd7154c6989d-cert\") pod \"ingress-canary-2xl48\" (UID: \"e4f8c700-e1c0-4abc-bda7-cd7154c6989d\") " pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202815    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.202867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdlbh\" (UniqueName: \"kubernetes.io/projected/e4f8c700-e1c0-4abc-bda7-cd7154c6989d-kube-api-access-xdlbh\") pod \"ingress-canary-2xl48\" (UID: \"e4f8c700-e1c0-4abc-bda7-cd7154c6989d\") " pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-ca-trust-extracted\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202897    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5478bd7-3b16-49c6-a902-e016f8f4e82d-tmp-dir\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202927    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-trusted-ca\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.202945    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.202959    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-image-registry-private-configuration\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.202962    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:16.203196 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.203108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-installation-pull-secrets\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.203555 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.203334    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-ca-trust-extracted\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.203555 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.203428    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-certificates\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.203616 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.203594    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:09:16.703573947 +0000 UTC m=+36.594604452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:16.203725 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.203706    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-trusted-ca\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.206887 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.206861    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-installation-pull-secrets\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.206973 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.206900    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-image-registry-private-configuration\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.215825 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.215792    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-bound-sa-token\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.216442 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.216422    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pxp9\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-kube-api-access-5pxp9\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.304227 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.304184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.304227 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.304231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5478bd7-3b16-49c6-a902-e016f8f4e82d-config-volume\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.304459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.304308    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svmwf\" (UniqueName: \"kubernetes.io/projected/c5478bd7-3b16-49c6-a902-e016f8f4e82d-kube-api-access-svmwf\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.304459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.304330    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4f8c700-e1c0-4abc-bda7-cd7154c6989d-cert\") pod \"ingress-canary-2xl48\" (UID: \"e4f8c700-e1c0-4abc-bda7-cd7154c6989d\") " pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.304459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.304376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdlbh\" (UniqueName: \"kubernetes.io/projected/e4f8c700-e1c0-4abc-bda7-cd7154c6989d-kube-api-access-xdlbh\") pod \"ingress-canary-2xl48\" (UID: \"e4f8c700-e1c0-4abc-bda7-cd7154c6989d\") " pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.304459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.304414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5478bd7-3b16-49c6-a902-e016f8f4e82d-tmp-dir\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.304652 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.304411    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:16.304958 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.304939    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:09:16.804914754 +0000 UTC m=+36.695945245 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:16.305152 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.305039    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5478bd7-3b16-49c6-a902-e016f8f4e82d-tmp-dir\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.305269 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.305191    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5478bd7-3b16-49c6-a902-e016f8f4e82d-config-volume\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.307351 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.307321    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e4f8c700-e1c0-4abc-bda7-cd7154c6989d-cert\") pod \"ingress-canary-2xl48\" (UID: \"e4f8c700-e1c0-4abc-bda7-cd7154c6989d\") " pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.316074 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.316022    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svmwf\" (UniqueName: \"kubernetes.io/projected/c5478bd7-3b16-49c6-a902-e016f8f4e82d-kube-api-access-svmwf\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.316754 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.316732    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdlbh\" (UniqueName: \"kubernetes.io/projected/e4f8c700-e1c0-4abc-bda7-cd7154c6989d-kube-api-access-xdlbh\") pod \"ingress-canary-2xl48\" (UID: \"e4f8c700-e1c0-4abc-bda7-cd7154c6989d\") " pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.320487 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.320454    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r76ff_59036915-3574-439b-bdd6-11bbde0bc1dc/konnectivity-agent/0.log" limit=1024
Jul 09 22:09:16.395561 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.395527    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2xl48"
Jul 09 22:09:16.469156 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.469116    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-r76ff_59036915-3574-439b-bdd6-11bbde0bc1dc/konnectivity-agent/0.log" limit=1024
Jul 09 22:09:16.615301 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.615190    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2xl48"]
Jul 09 22:09:16.620963 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:09:16.620935    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f8c700_e1c0_4abc_bda7_cd7154c6989d.slice/crio-2b2b38335e458db17d9d7c5f8566868327bb7826467bbacb9d51e5c8999207fb WatchSource:0}: Error finding container 2b2b38335e458db17d9d7c5f8566868327bb7826467bbacb9d51e5c8999207fb: Status 404 returned error can't find the container with id 2b2b38335e458db17d9d7c5f8566868327bb7826467bbacb9d51e5c8999207fb
Jul 09 22:09:16.674137 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.674098    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:16.674329 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.674155    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:16.674329 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.674153    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:16.680901 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.680872    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 22:09:16.681036 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.680927    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 22:09:16.681116 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.681094    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zm9fl\""
Jul 09 22:09:16.681358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.681339    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 22:09:16.681467 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.681410    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 22:09:16.681534 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.681495    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k7bfs\""
Jul 09 22:09:16.712283 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.712234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:16.712434 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.712397    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:16.712434 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.712413    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:16.712511 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.712476    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:09:17.712457548 +0000 UTC m=+37.603488054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:16.813429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.813390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:16.813608 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.813514    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:16.813608 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:16.813587    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:09:17.813566653 +0000 UTC m=+37.704597152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:16.818651 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:16.818616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xl48" event={"ID":"e4f8c700-e1c0-4abc-bda7-cd7154c6989d","Type":"ContainerStarted","Data":"2b2b38335e458db17d9d7c5f8566868327bb7826467bbacb9d51e5c8999207fb"}
Jul 09 22:09:17.719555 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:17.719519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:17.720320 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:17.719628    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:17.720320 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:17.719639    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:17.720320 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:17.719685    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:09:19.719672478 +0000 UTC m=+39.610702971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:17.820240 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:17.819989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:17.820436 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:17.820102    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:17.820436 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:17.820363    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:09:19.82033889 +0000 UTC m=+39.711369398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:19.734764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:19.734728    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:19.735330 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:19.734858    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:19.735330 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:19.734877    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:19.735330 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:19.734938    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:09:23.734923449 +0000 UTC m=+43.625953941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:19.835840 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:19.835796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:19.836047 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:19.835962    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:19.836047 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:19.836043    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:09:23.836023796 +0000 UTC m=+43.727054289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:22.833362 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:22.833275    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2xl48" event={"ID":"e4f8c700-e1c0-4abc-bda7-cd7154c6989d","Type":"ContainerStarted","Data":"bec13b9ba65612f827d91e27e3331580e4fbecfb70f559b491c6ee30ca53e0cd"}
Jul 09 22:09:22.835888 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:22.835863    2566 generic.go:356] "Generic (PLEG): container finished" podID="da8369e8-7d31-4ff5-a807-ba6165b0b90b" containerID="2de68da85294b7b647af3ce403ae3812c13bc1496da210b7b3e69f529578be8b" exitCode=0
Jul 09 22:09:22.836003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:22.835918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerDied","Data":"2de68da85294b7b647af3ce403ae3812c13bc1496da210b7b3e69f529578be8b"}
Jul 09 22:09:22.861935 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:22.861892    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2xl48" podStartSLOduration=1.5685819250000002 podStartE2EDuration="6.861868789s" totalImagesPullingTime="5.293286864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:22.861729882 +0000 UTC m=+42.752760408" watchObservedRunningTime="2026-07-09 22:09:22.861868789 +0000 UTC m=+42.752899303"
Jul 09 22:09:23.769543 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:23.769502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:23.769737 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:23.769655    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:23.769737 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:23.769675    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:23.769737 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:23.769732    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:09:31.769714511 +0000 UTC m=+51.660745002 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:23.841342 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:23.841303    2566 generic.go:356] "Generic (PLEG): container finished" podID="da8369e8-7d31-4ff5-a807-ba6165b0b90b" containerID="71945ae1b7b11f9459b83b873353bf6c0d384ebde0564bf75fcb22edf073ba8a" exitCode=0
Jul 09 22:09:23.841787 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:23.841394    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerDied","Data":"71945ae1b7b11f9459b83b873353bf6c0d384ebde0564bf75fcb22edf073ba8a"}
Jul 09 22:09:23.870332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:23.870307    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:23.870498 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:23.870475    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:23.870569 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:23.870558    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:09:31.870537818 +0000 UTC m=+51.761568318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:24.846857 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:24.846820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8znkb" event={"ID":"da8369e8-7d31-4ff5-a807-ba6165b0b90b","Type":"ContainerStarted","Data":"026ce898c5c93d379b94511b74c0352c8c6ddd45f4178cb93bbb3c5f05e43bb3"}
Jul 09 22:09:24.894688 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:24.894629    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8znkb" podStartSLOduration=8.336350949 podStartE2EDuration="42.894612341s" totalImagesPullingTime="34.558261392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:24.885959436 +0000 UTC m=+44.776989950" watchObservedRunningTime="2026-07-09 22:09:24.894612341 +0000 UTC m=+44.785642887"
Jul 09 22:09:31.827937 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:31.827874    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:31.828374 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:31.828029    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:31.828374 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:31.828051    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:31.828374 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:31.828117    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:09:47.828101546 +0000 UTC m=+67.719132038 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:31.928994 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:31.928951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:31.929189 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:31.929115    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:31.929189 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:31.929178    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:09:47.929161883 +0000 UTC m=+67.820192374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:41.814292 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:41.814263    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r24c5"
Jul 09 22:09:46.940929 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:46.940898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:09:46.941404 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:46.940965    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:46.944513 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:46.944491    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 22:09:46.944561 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:46.944511    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 22:09:46.951462 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:46.951441    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 22:09:46.951533 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:46.951505    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:10:50.951489907 +0000 UTC m=+130.842520399 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : secret "metrics-daemon-secret" not found
Jul 09 22:09:46.954566 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:46.954543    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/399349ba-0655-4074-b684-4e43c64ddfe3-original-pull-secret\") pod \"global-pull-secret-syncer-dx7b8\" (UID: \"399349ba-0655-4074-b684-4e43c64ddfe3\") " pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:46.987431 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:46.987407    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dx7b8"
Jul 09 22:09:47.041354 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.041323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:47.044368 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.044339    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 22:09:47.054087 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.054062    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 22:09:47.065082 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.065057    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6xl7\" (UniqueName: \"kubernetes.io/projected/2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f-kube-api-access-v6xl7\") pod \"network-check-target-q87cw\" (UID: \"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f\") " pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:47.111401 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.111372    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dx7b8"]
Jul 09 22:09:47.116756 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:09:47.116727    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399349ba_0655_4074_b684_4e43c64ddfe3.slice/crio-9f0f485a179108394906793c8a905622f48f18cf4efa26b4487ebbbdadaeb055 WatchSource:0}: Error finding container 9f0f485a179108394906793c8a905622f48f18cf4efa26b4487ebbbdadaeb055: Status 404 returned error can't find the container with id 9f0f485a179108394906793c8a905622f48f18cf4efa26b4487ebbbdadaeb055
Jul 09 22:09:47.303423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.303389    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k7bfs\""
Jul 09 22:09:47.311259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.311232    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:47.429666 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.429633    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q87cw"]
Jul 09 22:09:47.432893 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:09:47.432868    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3d9ac2_796e_4e74_978c_f7dc81f3bb4f.slice/crio-7ca8c5983a1bcc1dde16982d7a1703331656ccacfafbac54ecc643ddf27f8f0a WatchSource:0}: Error finding container 7ca8c5983a1bcc1dde16982d7a1703331656ccacfafbac54ecc643ddf27f8f0a: Status 404 returned error can't find the container with id 7ca8c5983a1bcc1dde16982d7a1703331656ccacfafbac54ecc643ddf27f8f0a
Jul 09 22:09:47.847074 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.847041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:09:47.847315 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:47.847177    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:09:47.847315 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:47.847191    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:09:47.847315 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:47.847301    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:10:19.847279415 +0000 UTC m=+99.738309922 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:09:47.904560 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.904520    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q87cw" event={"ID":"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f","Type":"ContainerStarted","Data":"7ca8c5983a1bcc1dde16982d7a1703331656ccacfafbac54ecc643ddf27f8f0a"}
Jul 09 22:09:47.905813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.905781    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dx7b8" event={"ID":"399349ba-0655-4074-b684-4e43c64ddfe3","Type":"ContainerStarted","Data":"9f0f485a179108394906793c8a905622f48f18cf4efa26b4487ebbbdadaeb055"}
Jul 09 22:09:47.948378 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:47.948344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:09:47.948817 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:47.948509    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:09:47.948817 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:09:47.948575    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:10:19.948558847 +0000 UTC m=+99.839589342 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:09:52.920726 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:52.920682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q87cw" event={"ID":"2d3d9ac2-796e-4e74-978c-f7dc81f3bb4f","Type":"ContainerStarted","Data":"b0ee77f5d7a0f09fe50be26f2b7ff677f6bf646f9844408a5835a707dbe2a989"}
Jul 09 22:09:52.921202 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:52.920787    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:09:52.922100 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:52.922075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dx7b8" event={"ID":"399349ba-0655-4074-b684-4e43c64ddfe3","Type":"ContainerStarted","Data":"581dbceae7688a3bf0e3467385c15a2e04f892be6a130614521f00f4e8df227a"}
Jul 09 22:09:52.939750 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:52.939700    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q87cw" podStartSLOduration=65.998006121 podStartE2EDuration="1m10.939684784s" totalImagesPullingTime="4.941678663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:52.938970957 +0000 UTC m=+72.830001471" watchObservedRunningTime="2026-07-09 22:09:52.939684784 +0000 UTC m=+72.830715297"
Jul 09 22:09:52.955949 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:09:52.955889    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dx7b8" podStartSLOduration=65.696773909 podStartE2EDuration="1m10.955870055s" totalImagesPullingTime="5.259096146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:09:52.955481348 +0000 UTC m=+72.846511864" watchObservedRunningTime="2026-07-09 22:09:52.955870055 +0000 UTC m=+72.846900570"
Jul 09 22:10:19.887617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:10:19.887579    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:10:19.888119 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:19.887755    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:10:19.888119 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:19.887773    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:10:19.888119 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:19.887852    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:11:23.887834407 +0000 UTC m=+163.778864900 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:10:19.988964 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:10:19.988926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:10:19.989323 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:19.989053    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:10:19.989323 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:19.989118    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:11:23.989100924 +0000 UTC m=+163.880131426 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:10:23.927337 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:10:23.927301    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q87cw"
Jul 09 22:10:51.002035 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:10:51.001993    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:10:51.002585 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:51.002136    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 22:10:51.002585 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:10:51.002216    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:12:53.002198256 +0000 UTC m=+252.893228749 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : secret "metrics-daemon-secret" not found
Jul 09 22:11:19.014833 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:19.014775    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff"
Jul 09 22:11:19.069129 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:19.069073    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-g9qfh" podUID="c5478bd7-3b16-49c6-a902-e016f8f4e82d"
Jul 09 22:11:19.125350 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:11:19.125313    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:11:19.125350 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:11:19.125341    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:11:19.694189 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:19.694144    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:11:23.933106 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:11:23.933069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:11:23.933571 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:23.933189    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 22:11:23.933571 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:23.933202    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cc865548f-9rtph: secret "image-registry-tls" not found
Jul 09 22:11:23.933571 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:23.933284    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls podName:ed48d949-2eb1-43bd-8c4f-e2e163a299ff nodeName:}" failed. No retries permitted until 2026-07-09 22:13:25.933264603 +0000 UTC m=+285.824295109 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls") pod "image-registry-5cc865548f-9rtph" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff") : secret "image-registry-tls" not found
Jul 09 22:11:24.033891 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:11:24.033854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:11:24.034074 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:24.033982    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 22:11:24.034074 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:11:24.034063    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls podName:c5478bd7-3b16-49c6-a902-e016f8f4e82d nodeName:}" failed. No retries permitted until 2026-07-09 22:13:26.034028018 +0000 UTC m=+285.925058510 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls") pod "dns-default-g9qfh" (UID: "c5478bd7-3b16-49c6-a902-e016f8f4e82d") : secret "dns-default-metrics-tls" not found
Jul 09 22:11:31.675235 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:11:31.675195    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:12:53.049583 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:53.049538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:12:53.050182 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:53.049719    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 22:12:53.050182 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:53.049816    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs podName:96332960-3e27-4437-b375-7c16a4a8a2ea nodeName:}" failed. No retries permitted until 2026-07-09 22:14:55.04979367 +0000 UTC m=+374.940824182 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs") pod "network-metrics-daemon-8xh4f" (UID: "96332960-3e27-4437-b375-7c16a4a8a2ea") : secret "metrics-daemon-secret" not found
Jul 09 22:12:55.951702 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.951663    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"]
Jul 09 22:12:55.953921 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.953898    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:55.958078 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:55.958042    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Jul 09 22:12:55.958609 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:55.958581    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Jul 09 22:12:55.958789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.958770    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-mbc8s"]
Jul 09 22:12:55.959079 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:55.959054    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-monitoring-operator-dockercfg-jzcz5\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jzcz5\"" type="*v1.Secret"
Jul 09 22:12:55.959370 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:55.959345    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"telemetry-config\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" type="*v1.ConfigMap"
Jul 09 22:12:55.959553 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:55.959529    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"cluster-monitoring-operator-64786d9cbb-ngfns\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" podUID="35412c7d-cb46-4257-857e-cdd8156e34aa" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:55.959847 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:55.959818    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-monitoring-operator-tls\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" type="*v1.Secret"
Jul 09 22:12:55.960704 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.960687    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.963695 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.963668    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 09 22:12:55.963695 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.963692    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 22:12:55.963810 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.963713    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mnp8s\""
Jul 09 22:12:55.963810 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.963670    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 09 22:12:55.963990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.963977    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 22:12:55.970270 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970237    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecd5587b-22e2-4d24-a201-e37a074f83ad-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.970357 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmkw\" (UniqueName: \"kubernetes.io/projected/35412c7d-cb46-4257-857e-cdd8156e34aa-kube-api-access-zkmkw\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:55.970357 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecd5587b-22e2-4d24-a201-e37a074f83ad-tmp\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.970471 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970359    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ecd5587b-22e2-4d24-a201-e37a074f83ad-snapshots\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.970471 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:55.970471 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecd5587b-22e2-4d24-a201-e37a074f83ad-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.970471 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr6tg\" (UniqueName: \"kubernetes.io/projected/ecd5587b-22e2-4d24-a201-e37a074f83ad-kube-api-access-fr6tg\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.970633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970501    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecd5587b-22e2-4d24-a201-e37a074f83ad-serving-cert\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:55.970633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.970526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/35412c7d-cb46-4257-857e-cdd8156e34aa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:55.978929 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.978888    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"]
Jul 09 22:12:55.980803 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.980760    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 09 22:12:55.981311 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.981296    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"]
Jul 09 22:12:55.981422 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.981408    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:55.982967 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.982950    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:55.990998 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.990975    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 09 22:12:55.991270 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.991237    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pqpng\""
Jul 09 22:12:55.991538 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.991523    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 09 22:12:55.991621 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.991527    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 09 22:12:55.991677 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.991545    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 09 22:12:55.991677 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.991629    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 09 22:12:55.991867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.991852    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-44q4n\""
Jul 09 22:12:55.993680 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:55.993664    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 09 22:12:56.002360 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.002337    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"]
Jul 09 22:12:56.012275 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.012235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"]
Jul 09 22:12:56.013399 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.013378    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-mbc8s"]
Jul 09 22:12:56.014385 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.014365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"]
Jul 09 22:12:56.048892 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.048861    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"]
Jul 09 22:12:56.051058 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.051037    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"
Jul 09 22:12:56.054506 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.054485    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-j9xct\""
Jul 09 22:12:56.054615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.054526    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 09 22:12:56.056070 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.056049    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 09 22:12:56.059545 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.059516    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"]
Jul 09 22:12:56.061403 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.061388    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"
Jul 09 22:12:56.063429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.063412    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jcdmx\""
Jul 09 22:12:56.069501 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.069480    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"]
Jul 09 22:12:56.071381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071363    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr6tg\" (UniqueName: \"kubernetes.io/projected/ecd5587b-22e2-4d24-a201-e37a074f83ad-kube-api-access-fr6tg\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.071488 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgwr5\" (UniqueName: \"kubernetes.io/projected/727b044e-47c9-4c8d-a75d-38d2b3d14cf2-kube-api-access-wgwr5\") pod \"network-check-source-989f8b8dc-8kt55\" (UID: \"727b044e-47c9-4c8d-a75d-38d2b3d14cf2\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"
Jul 09 22:12:56.071488 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071441    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cd47921f-23a4-46c0-8887-ce189ee42315-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:56.071488 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071467    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5pf6\" (UniqueName: \"kubernetes.io/projected/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-kube-api-access-w5pf6\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.071643 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecd5587b-22e2-4d24-a201-e37a074f83ad-tmp\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.071643 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071546    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecd5587b-22e2-4d24-a201-e37a074f83ad-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.071643 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071594    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:56.071643 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecd5587b-22e2-4d24-a201-e37a074f83ad-serving-cert\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.071820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071651    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/35412c7d-cb46-4257-857e-cdd8156e34aa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:56.071820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecd5587b-22e2-4d24-a201-e37a074f83ad-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.071820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071704    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db49n\" (UniqueName: \"kubernetes.io/projected/5bef2150-f55b-4757-b75e-8c27464c6a5e-kube-api-access-db49n\") pod \"volume-data-source-validator-5d456fdbcd-ljxg8\" (UID: \"5bef2150-f55b-4757-b75e-8c27464c6a5e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"
Jul 09 22:12:56.071820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.071820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071792    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:56.072029 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071828    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.072029 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071855    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecd5587b-22e2-4d24-a201-e37a074f83ad-tmp\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.072029 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmkw\" (UniqueName: \"kubernetes.io/projected/35412c7d-cb46-4257-857e-cdd8156e34aa-kube-api-access-zkmkw\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:56.072029 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.071928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ecd5587b-22e2-4d24-a201-e37a074f83ad-snapshots\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.072239 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.072219    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecd5587b-22e2-4d24-a201-e37a074f83ad-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.072566 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.072539    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ecd5587b-22e2-4d24-a201-e37a074f83ad-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.072626 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.072598    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ecd5587b-22e2-4d24-a201-e37a074f83ad-snapshots\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.074140 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.074121    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ecd5587b-22e2-4d24-a201-e37a074f83ad-serving-cert\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.082760 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.082727    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"]
Jul 09 22:12:56.096056 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.096025    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr6tg\" (UniqueName: \"kubernetes.io/projected/ecd5587b-22e2-4d24-a201-e37a074f83ad-kube-api-access-fr6tg\") pod \"insights-operator-5c847dcfc4-mbc8s\" (UID: \"ecd5587b-22e2-4d24-a201-e37a074f83ad\") " pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.159854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.159817    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"]
Jul 09 22:12:56.162454 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.162431    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.166558 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.166321    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 09 22:12:56.166558 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.166337    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 09 22:12:56.166558 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.166381    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 09 22:12:56.166558 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.166337    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rzz5n\""
Jul 09 22:12:56.166931 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.166856    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 09 22:12:56.172672 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172651    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgwr5\" (UniqueName: \"kubernetes.io/projected/727b044e-47c9-4c8d-a75d-38d2b3d14cf2-kube-api-access-wgwr5\") pod \"network-check-source-989f8b8dc-8kt55\" (UID: \"727b044e-47c9-4c8d-a75d-38d2b3d14cf2\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"
Jul 09 22:12:56.172794 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cd47921f-23a4-46c0-8887-ce189ee42315-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:56.172794 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172702    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5pf6\" (UniqueName: \"kubernetes.io/projected/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-kube-api-access-w5pf6\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.172794 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172731    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx4sz\" (UniqueName: \"kubernetes.io/projected/edc94141-8082-4394-a322-f66937bd7fb1-kube-api-access-qx4sz\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.172955 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/edc94141-8082-4394-a322-f66937bd7fb1-config\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.172955 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172910    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/edc94141-8082-4394-a322-f66937bd7fb1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.173055 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172962    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db49n\" (UniqueName: \"kubernetes.io/projected/5bef2150-f55b-4757-b75e-8c27464c6a5e-kube-api-access-db49n\") pod \"volume-data-source-validator-5d456fdbcd-ljxg8\" (UID: \"5bef2150-f55b-4757-b75e-8c27464c6a5e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"
Jul 09 22:12:56.173055 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.172992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.173055 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.173021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:56.173055 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.173046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.173263 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:56.173106    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 22:12:56.173263 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:56.173163    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert podName:cd47921f-23a4-46c0-8887-ce189ee42315 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:56.67314676 +0000 UTC m=+256.564177260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xx6fx" (UID: "cd47921f-23a4-46c0-8887-ce189ee42315") : secret "networking-console-plugin-cert" not found
Jul 09 22:12:56.173388 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.173367    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cd47921f-23a4-46c0-8887-ce189ee42315-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:56.173613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.173597    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.175113 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.175094    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.177762 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.177742    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"]
Jul 09 22:12:56.190261 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.190232    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgwr5\" (UniqueName: \"kubernetes.io/projected/727b044e-47c9-4c8d-a75d-38d2b3d14cf2-kube-api-access-wgwr5\") pod \"network-check-source-989f8b8dc-8kt55\" (UID: \"727b044e-47c9-4c8d-a75d-38d2b3d14cf2\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"
Jul 09 22:12:56.194763 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.194738    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5pf6\" (UniqueName: \"kubernetes.io/projected/11d804de-7aa9-4a6b-a90d-a9b10ef6f463-kube-api-access-w5pf6\") pod \"kube-storage-version-migrator-operator-846bf668f7-br6d2\" (UID: \"11d804de-7aa9-4a6b-a90d-a9b10ef6f463\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.195031 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.195012    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db49n\" (UniqueName: \"kubernetes.io/projected/5bef2150-f55b-4757-b75e-8c27464c6a5e-kube-api-access-db49n\") pod \"volume-data-source-validator-5d456fdbcd-ljxg8\" (UID: \"5bef2150-f55b-4757-b75e-8c27464c6a5e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"
Jul 09 22:12:56.269736 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.269700    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s"
Jul 09 22:12:56.274329 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.274308    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx4sz\" (UniqueName: \"kubernetes.io/projected/edc94141-8082-4394-a322-f66937bd7fb1-kube-api-access-qx4sz\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.274397 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.274357    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/edc94141-8082-4394-a322-f66937bd7fb1-config\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.274397 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.274374    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/edc94141-8082-4394-a322-f66937bd7fb1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.274896 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.274869    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/edc94141-8082-4394-a322-f66937bd7fb1-config\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.276590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.276569    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/edc94141-8082-4394-a322-f66937bd7fb1-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.282747 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.282725    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx4sz\" (UniqueName: \"kubernetes.io/projected/edc94141-8082-4394-a322-f66937bd7fb1-kube-api-access-qx4sz\") pod \"service-ca-operator-5d46d9ffd4-9s76j\" (UID: \"edc94141-8082-4394-a322-f66937bd7fb1\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.296058 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.296036    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"
Jul 09 22:12:56.360799 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.360540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"
Jul 09 22:12:56.369689 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.369662    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"
Jul 09 22:12:56.408657 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.408591    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-mbc8s"]
Jul 09 22:12:56.439289 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.439214    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2"]
Jul 09 22:12:56.442275 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:12:56.442237    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d804de_7aa9_4a6b_a90d_a9b10ef6f463.slice/crio-9322aa8713d49e9ec11fa536574234f8336f838d5ea8eb6ed5c55ce56e7f2878 WatchSource:0}: Error finding container 9322aa8713d49e9ec11fa536574234f8336f838d5ea8eb6ed5c55ce56e7f2878: Status 404 returned error can't find the container with id 9322aa8713d49e9ec11fa536574234f8336f838d5ea8eb6ed5c55ce56e7f2878
Jul 09 22:12:56.472126 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.472088    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"
Jul 09 22:12:56.502836 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.502796    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8"]
Jul 09 22:12:56.506374 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:12:56.506348    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bef2150_f55b_4757_b75e_8c27464c6a5e.slice/crio-500359cc9504061ab2d5c460a5bd2155b5a896aab1b2c1493eaa7ce08b0e2d26 WatchSource:0}: Error finding container 500359cc9504061ab2d5c460a5bd2155b5a896aab1b2c1493eaa7ce08b0e2d26: Status 404 returned error can't find the container with id 500359cc9504061ab2d5c460a5bd2155b5a896aab1b2c1493eaa7ce08b0e2d26
Jul 09 22:12:56.520864 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.520747    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55"]
Jul 09 22:12:56.524447 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:12:56.524397    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod727b044e_47c9_4c8d_a75d_38d2b3d14cf2.slice/crio-c9838d0fd8ab1ab23d42cd9907f4410b15142546e6a704a77d396cd916f37fb0 WatchSource:0}: Error finding container c9838d0fd8ab1ab23d42cd9907f4410b15142546e6a704a77d396cd916f37fb0: Status 404 returned error can't find the container with id c9838d0fd8ab1ab23d42cd9907f4410b15142546e6a704a77d396cd916f37fb0
Jul 09 22:12:56.614548 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.611995    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j"]
Jul 09 22:12:56.615259 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:12:56.615130    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc94141_8082_4394_a322_f66937bd7fb1.slice/crio-4fbb655f3c58659ce35349b9eb08840db0e13aff5fc331b6c8a125b9ca0dddc1 WatchSource:0}: Error finding container 4fbb655f3c58659ce35349b9eb08840db0e13aff5fc331b6c8a125b9ca0dddc1: Status 404 returned error can't find the container with id 4fbb655f3c58659ce35349b9eb08840db0e13aff5fc331b6c8a125b9ca0dddc1
Jul 09 22:12:56.678952 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.678908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:56.679125 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:56.679062    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 22:12:56.679165 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:56.679140    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert podName:cd47921f-23a4-46c0-8887-ce189ee42315 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.679123591 +0000 UTC m=+257.570154083 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xx6fx" (UID: "cd47921f-23a4-46c0-8887-ce189ee42315") : secret "networking-console-plugin-cert" not found
Jul 09 22:12:56.802683 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:56.802590    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 22:12:57.031356 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.031324    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 22:12:57.041444 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.041418    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmkw\" (UniqueName: \"kubernetes.io/projected/35412c7d-cb46-4257-857e-cdd8156e34aa-kube-api-access-zkmkw\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:57.072450 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.072387    2566 configmap.go:193] Couldn't get configMap openshift-monitoring/telemetry-config: failed to sync configmap cache: timed out waiting for the condition
Jul 09 22:12:57.072592 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.072459    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/35412c7d-cb46-4257-857e-cdd8156e34aa-telemetry-config podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.572442663 +0000 UTC m=+257.463473155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "telemetry-config" (UniqueName: "kubernetes.io/configmap/35412c7d-cb46-4257-857e-cdd8156e34aa-telemetry-config") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : failed to sync configmap cache: timed out waiting for the condition
Jul 09 22:12:57.072592 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.072390    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: failed to sync secret cache: timed out waiting for the condition
Jul 09 22:12:57.072592 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.072526    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:12:57.572511807 +0000 UTC m=+257.463542303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : failed to sync secret cache: timed out waiting for the condition
Jul 09 22:12:57.130726 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.130692    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 09 22:12:57.359565 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.359476    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55" event={"ID":"727b044e-47c9-4c8d-a75d-38d2b3d14cf2","Type":"ContainerStarted","Data":"84d70fa7d99836cf1cfee287caae1b0778e4f69c4507c212d06d1ee454af8d14"}
Jul 09 22:12:57.359565 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.359540    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55" event={"ID":"727b044e-47c9-4c8d-a75d-38d2b3d14cf2","Type":"ContainerStarted","Data":"c9838d0fd8ab1ab23d42cd9907f4410b15142546e6a704a77d396cd916f37fb0"}
Jul 09 22:12:57.360849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.360806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j" event={"ID":"edc94141-8082-4394-a322-f66937bd7fb1","Type":"ContainerStarted","Data":"4fbb655f3c58659ce35349b9eb08840db0e13aff5fc331b6c8a125b9ca0dddc1"}
Jul 09 22:12:57.361961 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.361932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2" event={"ID":"11d804de-7aa9-4a6b-a90d-a9b10ef6f463","Type":"ContainerStarted","Data":"9322aa8713d49e9ec11fa536574234f8336f838d5ea8eb6ed5c55ce56e7f2878"}
Jul 09 22:12:57.363048 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.363016    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8" event={"ID":"5bef2150-f55b-4757-b75e-8c27464c6a5e","Type":"ContainerStarted","Data":"500359cc9504061ab2d5c460a5bd2155b5a896aab1b2c1493eaa7ce08b0e2d26"}
Jul 09 22:12:57.364235 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.364208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s" event={"ID":"ecd5587b-22e2-4d24-a201-e37a074f83ad","Type":"ContainerStarted","Data":"9a89b39c4ad8a47f0d6efd70aa07fc9ab0f0dbf2675b272d8f4fce4c5f669a70"}
Jul 09 22:12:57.376035 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.375978    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-8kt55" podStartSLOduration=1.375961939 podStartE2EDuration="1.375961939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:12:57.375262294 +0000 UTC m=+257.266292803" watchObservedRunningTime="2026-07-09 22:12:57.375961939 +0000 UTC m=+257.266992453"
Jul 09 22:12:57.459542 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.459028    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 09 22:12:57.472564 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.472526    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-jzcz5\""
Jul 09 22:12:57.589127 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.588187    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:57.589127 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.588267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/35412c7d-cb46-4257-857e-cdd8156e34aa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:57.589127 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.588863    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 22:12:57.589127 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.588952    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:12:58.588927238 +0000 UTC m=+258.479957748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : secret "cluster-monitoring-operator-tls" not found
Jul 09 22:12:57.589127 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.589081    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/35412c7d-cb46-4257-857e-cdd8156e34aa-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:57.689622 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:57.689527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:57.689857 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.689677    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 22:12:57.689857 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:57.689754    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert podName:cd47921f-23a4-46c0-8887-ce189ee42315 nodeName:}" failed. No retries permitted until 2026-07-09 22:12:59.689733862 +0000 UTC m=+259.580764356 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xx6fx" (UID: "cd47921f-23a4-46c0-8887-ce189ee42315") : secret "networking-console-plugin-cert" not found
Jul 09 22:12:58.368497 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:58.368397    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8" event={"ID":"5bef2150-f55b-4757-b75e-8c27464c6a5e","Type":"ContainerStarted","Data":"648c62a099c33cf4f60fc83fecdf5c9db4cb902a83885a66e7b5ce512c333036"}
Jul 09 22:12:58.384535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:58.384471    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-ljxg8" podStartSLOduration=0.832366096 podStartE2EDuration="2.384448551s" totalImagesPullingTime="1.552082455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:12:58.383978721 +0000 UTC m=+258.275009236" watchObservedRunningTime="2026-07-09 22:12:58.384448551 +0000 UTC m=+258.275479066"
Jul 09 22:12:58.600876 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:58.600840    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:12:58.601035 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:58.600962    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 22:12:58.601035 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:58.601019    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:13:00.601003188 +0000 UTC m=+260.492033700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : secret "cluster-monitoring-operator-tls" not found
Jul 09 22:12:59.710587 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:12:59.710539    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:12:59.711071 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:59.710695    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 22:12:59.711071 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:12:59.710764    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert podName:cd47921f-23a4-46c0-8887-ce189ee42315 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.710746306 +0000 UTC m=+263.601776802 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xx6fx" (UID: "cd47921f-23a4-46c0-8887-ce189ee42315") : secret "networking-console-plugin-cert" not found
Jul 09 22:13:00.376994 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.376880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j" event={"ID":"edc94141-8082-4394-a322-f66937bd7fb1","Type":"ContainerStarted","Data":"52c1d17cfdcf9a8a57f1c4f4910ea823e159c98e8aadc4612e1843f9f4b669eb"}
Jul 09 22:13:00.384321 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.384276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2" event={"ID":"11d804de-7aa9-4a6b-a90d-a9b10ef6f463","Type":"ContainerStarted","Data":"d45d3e2f32bb9bb258ab02d361938b8d8eb824420e8568617a7a348c1c4053bf"}
Jul 09 22:13:00.386061 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.386023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s" event={"ID":"ecd5587b-22e2-4d24-a201-e37a074f83ad","Type":"ContainerStarted","Data":"3a9e892cf89e35a42a1dffb069f6cf1a8d5aa2988301943a1d33deb366e51d5d"}
Jul 09 22:13:00.393903 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.393824    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j" podStartSLOduration=0.91164243 podStartE2EDuration="4.393803144s" totalImagesPullingTime="3.482160714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:00.392890475 +0000 UTC m=+260.283920989" watchObservedRunningTime="2026-07-09 22:13:00.393803144 +0000 UTC m=+260.284833653"
Jul 09 22:13:00.409502 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.409260    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s" podStartSLOduration=1.7318292419999999 podStartE2EDuration="5.409225792s" totalImagesPullingTime="3.67739655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:00.408982682 +0000 UTC m=+260.300013193" watchObservedRunningTime="2026-07-09 22:13:00.409225792 +0000 UTC m=+260.300256308"
Jul 09 22:13:00.424625 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.424562    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2" podStartSLOduration=1.772244514 podStartE2EDuration="5.424542692s" totalImagesPullingTime="3.652298178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:00.424509488 +0000 UTC m=+260.315540003" watchObservedRunningTime="2026-07-09 22:13:00.424542692 +0000 UTC m=+260.315573207"
Jul 09 22:13:00.620525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:00.620469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:13:00.620733 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:00.620586    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 22:13:00.620733 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:00.620675    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:13:04.620658758 +0000 UTC m=+264.511689250 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : secret "cluster-monitoring-operator-tls" not found
Jul 09 22:13:01.805045 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.805007    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-br8qn"]
Jul 09 22:13:01.807691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.807667    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:01.811304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.811264    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 22:13:01.811304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.811292    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 22:13:01.811855 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.811830    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lm8tn\""
Jul 09 22:13:01.825203 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.825169    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-br8qn"]
Jul 09 22:13:01.932233 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.932138    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:01.932233 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.932210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/405b2834-9696-49e6-94e9-97c5c2e75963-data-volume\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:01.932436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.932265    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/405b2834-9696-49e6-94e9-97c5c2e75963-crio-socket\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:01.932436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.932295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwgsr\" (UniqueName: \"kubernetes.io/projected/405b2834-9696-49e6-94e9-97c5c2e75963-kube-api-access-rwgsr\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:01.932436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:01.932336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.032861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.032824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwgsr\" (UniqueName: \"kubernetes.io/projected/405b2834-9696-49e6-94e9-97c5c2e75963-kube-api-access-rwgsr\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.032861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.032869    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.033089 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.032990    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.033089 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.033039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/405b2834-9696-49e6-94e9-97c5c2e75963-data-volume\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.033089 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.032999    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:02.033182 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.033095    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 22:13:02.033182 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.033148    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:02.53312934 +0000 UTC m=+262.424159835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:02.033182 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.033164    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:02.533158003 +0000 UTC m=+262.424188494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : configmap "kube-rbac-proxy" not found
Jul 09 22:13:02.033182 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.033178    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/405b2834-9696-49e6-94e9-97c5c2e75963-crio-socket\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.033404 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.033271    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/405b2834-9696-49e6-94e9-97c5c2e75963-crio-socket\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.033404 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.033369    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/405b2834-9696-49e6-94e9-97c5c2e75963-data-volume\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.041626 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.041589    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwgsr\" (UniqueName: \"kubernetes.io/projected/405b2834-9696-49e6-94e9-97c5c2e75963-kube-api-access-rwgsr\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.170949 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.170862    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"]
Jul 09 22:13:02.173503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.173470    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"
Jul 09 22:13:02.177525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.176936    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t2sz8\""
Jul 09 22:13:02.177525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.177271    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 09 22:13:02.177525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.177420    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 09 22:13:02.183003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.182954    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"]
Jul 09 22:13:02.235212 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.235169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjptg\" (UniqueName: \"kubernetes.io/projected/7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc-kube-api-access-wjptg\") pod \"migrator-d57d58cc9-c42ff\" (UID: \"7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"
Jul 09 22:13:02.335835 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.335796    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjptg\" (UniqueName: \"kubernetes.io/projected/7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc-kube-api-access-wjptg\") pod \"migrator-d57d58cc9-c42ff\" (UID: \"7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"
Jul 09 22:13:02.344443 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.344412    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjptg\" (UniqueName: \"kubernetes.io/projected/7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc-kube-api-access-wjptg\") pod \"migrator-d57d58cc9-c42ff\" (UID: \"7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"
Jul 09 22:13:02.487969 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.487932    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"
Jul 09 22:13:02.537539 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.537503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.537704 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.537645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:02.537704 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.537670    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:02.537839 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.537750    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.537725122 +0000 UTC m=+263.428755627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:02.537839 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.537777    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 22:13:02.537953 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:02.537887    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:03.537841264 +0000 UTC m=+263.428871761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : configmap "kube-rbac-proxy" not found
Jul 09 22:13:02.615694 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:02.615654    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff"]
Jul 09 22:13:02.618418 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:02.618392    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d41b6bc_4f9d_4aa7_9c2b_30ddaa960bfc.slice/crio-d88888198df472516bef60cfe8fdd4612f89d43f5bca7e3f4faced1a5441a903 WatchSource:0}: Error finding container d88888198df472516bef60cfe8fdd4612f89d43f5bca7e3f4faced1a5441a903: Status 404 returned error can't find the container with id d88888198df472516bef60cfe8fdd4612f89d43f5bca7e3f4faced1a5441a903
Jul 09 22:13:03.396990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:03.396955    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff" event={"ID":"7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc","Type":"ContainerStarted","Data":"d88888198df472516bef60cfe8fdd4612f89d43f5bca7e3f4faced1a5441a903"}
Jul 09 22:13:03.544032 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:03.543988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:03.544209 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:03.544063    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:03.544209 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:03.544118    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 22:13:03.544209 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:03.544184    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:05.54416948 +0000 UTC m=+265.435199976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : configmap "kube-rbac-proxy" not found
Jul 09 22:13:03.544209 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:03.544204    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:03.544387 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:03.544298    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:05.544283807 +0000 UTC m=+265.435314300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:03.594292 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:03.594264    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2r5v5_87ff8432-417c-4f78-b181-4f44077cf60f/dns-node-resolver/0.log"
Jul 09 22:13:03.745514 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:03.745476    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:13:03.745702 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:03.745597    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 22:13:03.745702 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:03.745654    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert podName:cd47921f-23a4-46c0-8887-ce189ee42315 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:11.745639635 +0000 UTC m=+271.636670127 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xx6fx" (UID: "cd47921f-23a4-46c0-8887-ce189ee42315") : secret "networking-console-plugin-cert" not found
Jul 09 22:13:04.402277 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.402224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff" event={"ID":"7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc","Type":"ContainerStarted","Data":"822ed8b7d13f5857b8a9e2a8e823a4db23e6af664e797267270e4e955bf84a7c"}
Jul 09 22:13:04.402671 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.402294    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff" event={"ID":"7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc","Type":"ContainerStarted","Data":"5e87080386166c14e7c0128ceef970378e106c499fdf74507647a6983dd9b393"}
Jul 09 22:13:04.418617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.418565    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-c42ff" podStartSLOduration=0.756259074 podStartE2EDuration="2.418546933s" totalImagesPullingTime="1.662287859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:04.418100769 +0000 UTC m=+264.309131284" watchObservedRunningTime="2026-07-09 22:13:04.418546933 +0000 UTC m=+264.309577448"
Jul 09 22:13:04.438111 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.438069    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tjwnb"]
Jul 09 22:13:04.440196 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.440178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.442486 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.442450    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 09 22:13:04.442617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.442485    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 09 22:13:04.442617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.442456    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 09 22:13:04.442617 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.442577    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rxzpd\""
Jul 09 22:13:04.442777 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.442763    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 09 22:13:04.449724 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.449698    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tjwnb"]
Jul 09 22:13:04.552401 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.552305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77310b4d-af51-4e30-aa21-8d0b50bafa53-signing-cabundle\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.552401 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.552347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77310b4d-af51-4e30-aa21-8d0b50bafa53-signing-key\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.552607 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.552443    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pttfq\" (UniqueName: \"kubernetes.io/projected/77310b4d-af51-4e30-aa21-8d0b50bafa53-kube-api-access-pttfq\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.592693 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.592666    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8z5xq_94f487f9-70be-4b90-a26c-4f235797dd0f/node-ca/0.log"
Jul 09 22:13:04.653166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.653128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pttfq\" (UniqueName: \"kubernetes.io/projected/77310b4d-af51-4e30-aa21-8d0b50bafa53-kube-api-access-pttfq\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.653383 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.653185    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77310b4d-af51-4e30-aa21-8d0b50bafa53-signing-cabundle\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.653383 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.653211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77310b4d-af51-4e30-aa21-8d0b50bafa53-signing-key\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.653383 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.653241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:13:04.653545 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:04.653426    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 22:13:04.653545 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:04.653493    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:13:12.653474882 +0000 UTC m=+272.544505377 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : secret "cluster-monitoring-operator-tls" not found
Jul 09 22:13:04.653916 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.653893    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77310b4d-af51-4e30-aa21-8d0b50bafa53-signing-cabundle\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.655796 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.655777    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77310b4d-af51-4e30-aa21-8d0b50bafa53-signing-key\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.661589 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.661571    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pttfq\" (UniqueName: \"kubernetes.io/projected/77310b4d-af51-4e30-aa21-8d0b50bafa53-kube-api-access-pttfq\") pod \"service-ca-758f599c5-tjwnb\" (UID: \"77310b4d-af51-4e30-aa21-8d0b50bafa53\") " pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.750521 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.750479    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-tjwnb"
Jul 09 22:13:04.879216 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:04.877689    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-tjwnb"]
Jul 09 22:13:04.883395 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:04.883368    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77310b4d_af51_4e30_aa21_8d0b50bafa53.slice/crio-b2a5bed60cf6302f40c76a4cc0b98ebfcfaf3d3265b1feae4a985e1b982254e4 WatchSource:0}: Error finding container b2a5bed60cf6302f40c76a4cc0b98ebfcfaf3d3265b1feae4a985e1b982254e4: Status 404 returned error can't find the container with id b2a5bed60cf6302f40c76a4cc0b98ebfcfaf3d3265b1feae4a985e1b982254e4
Jul 09 22:13:05.394479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:05.394451    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2xl48_e4f8c700-e1c0-4abc-bda7-cd7154c6989d/serve-healthcheck-canary/0.log"
Jul 09 22:13:05.406906 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:05.406876    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-tjwnb" event={"ID":"77310b4d-af51-4e30-aa21-8d0b50bafa53","Type":"ContainerStarted","Data":"6b5ebfcefc45d01e9f6218abb56ae484c6548f79a85da1b6e5b80e7a381cf548"}
Jul 09 22:13:05.407372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:05.406923    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-tjwnb" event={"ID":"77310b4d-af51-4e30-aa21-8d0b50bafa53","Type":"ContainerStarted","Data":"b2a5bed60cf6302f40c76a4cc0b98ebfcfaf3d3265b1feae4a985e1b982254e4"}
Jul 09 22:13:05.425780 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:05.425732    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-tjwnb" podStartSLOduration=1.425717123 podStartE2EDuration="1.425717123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:05.425073108 +0000 UTC m=+265.316103623" watchObservedRunningTime="2026-07-09 22:13:05.425717123 +0000 UTC m=+265.316747636"
Jul 09 22:13:05.561620 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:05.561578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:05.561802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:05.561678    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:05.561802 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:05.561732    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:05.561802 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:05.561771    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 22:13:05.561802 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:05.561800    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:09.561784388 +0000 UTC m=+269.452814881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:05.561956 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:05.561815    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:09.56180858 +0000 UTC m=+269.452839072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : configmap "kube-rbac-proxy" not found
Jul 09 22:13:09.598177 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:09.598139    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:09.598597 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:09.598228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:09.598597 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:09.598312    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:09.598597 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:09.598328    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 22:13:09.598597 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:09.598377    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:17.598362643 +0000 UTC m=+277.489393135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : secret "insights-runtime-extractor-tls" not found
Jul 09 22:13:09.598597 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:09.598393    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm podName:405b2834-9696-49e6-94e9-97c5c2e75963 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:17.598386912 +0000 UTC m=+277.489417403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm") pod "insights-runtime-extractor-br8qn" (UID: "405b2834-9696-49e6-94e9-97c5c2e75963") : configmap "kube-rbac-proxy" not found
Jul 09 22:13:11.820874 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:11.820833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:13:11.821269 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:11.821020    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 22:13:11.821269 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:11.821086    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert podName:cd47921f-23a4-46c0-8887-ce189ee42315 nodeName:}" failed. No retries permitted until 2026-07-09 22:13:27.821068883 +0000 UTC m=+287.712099375 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xx6fx" (UID: "cd47921f-23a4-46c0-8887-ce189ee42315") : secret "networking-console-plugin-cert" not found
Jul 09 22:13:12.729661 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:12.729617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:13:12.729843 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:12.729817    2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 22:13:12.729948 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:12.729936    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls podName:35412c7d-cb46-4257-857e-cdd8156e34aa nodeName:}" failed. No retries permitted until 2026-07-09 22:13:28.729914937 +0000 UTC m=+288.620945435 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-ngfns" (UID: "35412c7d-cb46-4257-857e-cdd8156e34aa") : secret "cluster-monitoring-operator-tls" not found
Jul 09 22:13:17.670998 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:17.670941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:17.671459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:17.671095    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:17.671459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:17.671372    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/405b2834-9696-49e6-94e9-97c5c2e75963-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:17.673426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:17.673399    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/405b2834-9696-49e6-94e9-97c5c2e75963-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-br8qn\" (UID: \"405b2834-9696-49e6-94e9-97c5c2e75963\") " pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:17.718046 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:17.718006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-br8qn"
Jul 09 22:13:17.848561 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:17.848525    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-br8qn"]
Jul 09 22:13:17.851683 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:17.851661    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405b2834_9696_49e6_94e9_97c5c2e75963.slice/crio-6d377d1137b9d40c86b1ebb6f3c083adbc50cb928b8d46b33b453e63c45c310c WatchSource:0}: Error finding container 6d377d1137b9d40c86b1ebb6f3c083adbc50cb928b8d46b33b453e63c45c310c: Status 404 returned error can't find the container with id 6d377d1137b9d40c86b1ebb6f3c083adbc50cb928b8d46b33b453e63c45c310c
Jul 09 22:13:18.452316 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:18.452236    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br8qn" event={"ID":"405b2834-9696-49e6-94e9-97c5c2e75963","Type":"ContainerStarted","Data":"6d377d1137b9d40c86b1ebb6f3c083adbc50cb928b8d46b33b453e63c45c310c"}
Jul 09 22:13:22.126684 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:22.126641    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff"
Jul 09 22:13:22.126684 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:22.126670    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-g9qfh" podUID="c5478bd7-3b16-49c6-a902-e016f8f4e82d"
Jul 09 22:13:22.468928 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:22.468890    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br8qn" event={"ID":"405b2834-9696-49e6-94e9-97c5c2e75963","Type":"ContainerStarted","Data":"cc82841f76a358e3a4ec5855b8f5b4dc185878408b6fe27ac689a707f12ff411"}
Jul 09 22:13:22.468928 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:22.468929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:13:22.469188 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:22.468956    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br8qn" event={"ID":"405b2834-9696-49e6-94e9-97c5c2e75963","Type":"ContainerStarted","Data":"87e1154862ecc42b9206c84c2447ac65de6458d6228e4b337a4199b57942917e"}
Jul 09 22:13:22.469188 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:22.468908    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:13:25.481112 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:25.481074    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-br8qn" event={"ID":"405b2834-9696-49e6-94e9-97c5c2e75963","Type":"ContainerStarted","Data":"b8b54e288580c47b91d850c414a2cd0e402f170dca79b6c0f12078d31d1a2525"}
Jul 09 22:13:25.499169 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:25.499119    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-br8qn" podStartSLOduration=17.251112546 podStartE2EDuration="24.499102592s" totalImagesPullingTime="7.247990046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:13:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:25.497351328 +0000 UTC m=+285.388381842" watchObservedRunningTime="2026-07-09 22:13:25.499102592 +0000 UTC m=+285.390133106"
Jul 09 22:13:25.944753 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:25.944658    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:13:25.947041 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:25.947011    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"image-registry-5cc865548f-9rtph\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") " pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:13:26.045655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.045600    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:13:26.048035 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.048007    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5478bd7-3b16-49c6-a902-e016f8f4e82d-metrics-tls\") pod \"dns-default-g9qfh\" (UID: \"c5478bd7-3b16-49c6-a902-e016f8f4e82d\") " pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:13:26.073257 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.073213    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jg4kh\""
Jul 09 22:13:26.073758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.073744    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77st2\""
Jul 09 22:13:26.081330 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.081306    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:13:26.081449 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.081410    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:13:26.227534 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.227505    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cc865548f-9rtph"]
Jul 09 22:13:26.230765 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:26.230737    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded48d949_2eb1_43bd_8c4f_e2e163a299ff.slice/crio-a94337efbc3298f51d94682041e1d20eda52842748e3ba4cf885dcb49b98222a WatchSource:0}: Error finding container a94337efbc3298f51d94682041e1d20eda52842748e3ba4cf885dcb49b98222a: Status 404 returned error can't find the container with id a94337efbc3298f51d94682041e1d20eda52842748e3ba4cf885dcb49b98222a
Jul 09 22:13:26.243790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.243767    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g9qfh"]
Jul 09 22:13:26.246915 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:26.246819    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5478bd7_3b16_49c6_a902_e016f8f4e82d.slice/crio-d69c6c4042a3d3c782f8d53970e1fc2679681398f9d1d2736210f56325fdc45e WatchSource:0}: Error finding container d69c6c4042a3d3c782f8d53970e1fc2679681398f9d1d2736210f56325fdc45e: Status 404 returned error can't find the container with id d69c6c4042a3d3c782f8d53970e1fc2679681398f9d1d2736210f56325fdc45e
Jul 09 22:13:26.485590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.485503    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g9qfh" event={"ID":"c5478bd7-3b16-49c6-a902-e016f8f4e82d","Type":"ContainerStarted","Data":"d69c6c4042a3d3c782f8d53970e1fc2679681398f9d1d2736210f56325fdc45e"}
Jul 09 22:13:26.486885 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.486858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" event={"ID":"ed48d949-2eb1-43bd-8c4f-e2e163a299ff","Type":"ContainerStarted","Data":"b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d"}
Jul 09 22:13:26.487025 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.486907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" event={"ID":"ed48d949-2eb1-43bd-8c4f-e2e163a299ff","Type":"ContainerStarted","Data":"a94337efbc3298f51d94682041e1d20eda52842748e3ba4cf885dcb49b98222a"}
Jul 09 22:13:26.505052 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:26.504996    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" podStartSLOduration=284.504978816 podStartE2EDuration="4m44.504978816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:26.50377504 +0000 UTC m=+286.394805567" watchObservedRunningTime="2026-07-09 22:13:26.504978816 +0000 UTC m=+286.396009369"
Jul 09 22:13:27.490598 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:27.490565    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:13:27.862906 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:27.862819    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:13:27.865299 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:27.865266    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cd47921f-23a4-46c0-8887-ce189ee42315-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xx6fx\" (UID: \"cd47921f-23a4-46c0-8887-ce189ee42315\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:13:28.090440 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.090404    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"
Jul 09 22:13:28.429663 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.429556    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx"]
Jul 09 22:13:28.432386 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:28.432361    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd47921f_23a4_46c0_8887_ce189ee42315.slice/crio-3c1cce070c37be093d82515a5a4c165a564dd7991edd184395e743602f150eeb WatchSource:0}: Error finding container 3c1cce070c37be093d82515a5a4c165a564dd7991edd184395e743602f150eeb: Status 404 returned error can't find the container with id 3c1cce070c37be093d82515a5a4c165a564dd7991edd184395e743602f150eeb
Jul 09 22:13:28.498275 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.498210    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g9qfh" event={"ID":"c5478bd7-3b16-49c6-a902-e016f8f4e82d","Type":"ContainerStarted","Data":"053d5b340a13476c40a1c75c9abdd1d13a228ac95b57d1f48a9ec9d9d876442a"}
Jul 09 22:13:28.499525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.499494    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx" event={"ID":"cd47921f-23a4-46c0-8887-ce189ee42315","Type":"ContainerStarted","Data":"3c1cce070c37be093d82515a5a4c165a564dd7991edd184395e743602f150eeb"}
Jul 09 22:13:28.771845 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.771813    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:13:28.774210 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.774178    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/35412c7d-cb46-4257-857e-cdd8156e34aa-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-ngfns\" (UID: \"35412c7d-cb46-4257-857e-cdd8156e34aa\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:13:28.963156 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:28.963117    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"
Jul 09 22:13:29.093622 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:29.093587    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns"]
Jul 09 22:13:29.097567 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:29.097537    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35412c7d_cb46_4257_857e_cdd8156e34aa.slice/crio-fe1233c525f845fc4b552cc54569c83130e8aaddc627e18c33894b005ab8a230 WatchSource:0}: Error finding container fe1233c525f845fc4b552cc54569c83130e8aaddc627e18c33894b005ab8a230: Status 404 returned error can't find the container with id fe1233c525f845fc4b552cc54569c83130e8aaddc627e18c33894b005ab8a230
Jul 09 22:13:29.505990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:29.505953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g9qfh" event={"ID":"c5478bd7-3b16-49c6-a902-e016f8f4e82d","Type":"ContainerStarted","Data":"afac3589d0f5d25e092b38a847cd9e681c1848f74828e194a5274b98037fa498"}
Jul 09 22:13:29.506500 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:29.506074    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:13:29.507258 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:29.507226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns" event={"ID":"35412c7d-cb46-4257-857e-cdd8156e34aa","Type":"ContainerStarted","Data":"fe1233c525f845fc4b552cc54569c83130e8aaddc627e18c33894b005ab8a230"}
Jul 09 22:13:30.511848 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:30.511813    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx" event={"ID":"cd47921f-23a4-46c0-8887-ce189ee42315","Type":"ContainerStarted","Data":"6817a96fd33df702dcb6d752477f687aab808aa143d6583ee3063d1c7e9ba08b"}
Jul 09 22:13:30.527783 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:30.527735    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xx6fx" podStartSLOduration=34.354428389 podStartE2EDuration="35.52772194s" totalImagesPullingTime="1.173293551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:30.527027783 +0000 UTC m=+290.418058280" watchObservedRunningTime="2026-07-09 22:13:30.52772194 +0000 UTC m=+290.418752453"
Jul 09 22:13:30.528280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:30.528230    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g9qfh" podStartSLOduration=252.428880969 podStartE2EDuration="4m14.528222313s" totalImagesPullingTime="2.099341344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:29.523735566 +0000 UTC m=+289.414766086" watchObservedRunningTime="2026-07-09 22:13:30.528222313 +0000 UTC m=+290.419252827"
Jul 09 22:13:31.516778 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.516745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns" event={"ID":"35412c7d-cb46-4257-857e-cdd8156e34aa","Type":"ContainerStarted","Data":"8d15320c2396c637b6d64eaadbfb98c9272e577c85966dc525609ed0bb40db01"}
Jul 09 22:13:31.531516 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.531451    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-ngfns" podStartSLOduration=34.198614705 podStartE2EDuration="36.531434129s" totalImagesPullingTime="2.332819424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:31.530912146 +0000 UTC m=+291.421942660" watchObservedRunningTime="2026-07-09 22:13:31.531434129 +0000 UTC m=+291.422464644"
Jul 09 22:13:31.924495 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.924459    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"]
Jul 09 22:13:31.926744 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.926725    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:31.928781 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.928754    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hwr2w\""
Jul 09 22:13:31.929007 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.928994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 22:13:31.941264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.937723    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"]
Jul 09 22:13:31.998891 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:31.998850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78ca84a6-3168-4db1-8425-574a9f7752fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5kfjk\" (UID: \"78ca84a6-3168-4db1-8425-574a9f7752fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:32.099357 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:32.099317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78ca84a6-3168-4db1-8425-574a9f7752fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5kfjk\" (UID: \"78ca84a6-3168-4db1-8425-574a9f7752fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:32.099548 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:32.099459    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 09 22:13:32.099548 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:32.099526    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78ca84a6-3168-4db1-8425-574a9f7752fe-tls-certificates podName:78ca84a6-3168-4db1-8425-574a9f7752fe nodeName:}" failed. No retries permitted until 2026-07-09 22:13:32.599509165 +0000 UTC m=+292.490539657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/78ca84a6-3168-4db1-8425-574a9f7752fe-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-5kfjk" (UID: "78ca84a6-3168-4db1-8425-574a9f7752fe") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 09 22:13:32.602679 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:32.602631    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78ca84a6-3168-4db1-8425-574a9f7752fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5kfjk\" (UID: \"78ca84a6-3168-4db1-8425-574a9f7752fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:32.605074 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:32.605048    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/78ca84a6-3168-4db1-8425-574a9f7752fe-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-5kfjk\" (UID: \"78ca84a6-3168-4db1-8425-574a9f7752fe\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:32.836263 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:32.836208    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:32.997280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:32.997224    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"]
Jul 09 22:13:33.000224 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:33.000193    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ca84a6_3168_4db1_8425_574a9f7752fe.slice/crio-60b1e5fc40e122eeca8aa6d659c5a8e69265927e71fbcf6e06bc5cc1b8ee161f WatchSource:0}: Error finding container 60b1e5fc40e122eeca8aa6d659c5a8e69265927e71fbcf6e06bc5cc1b8ee161f: Status 404 returned error can't find the container with id 60b1e5fc40e122eeca8aa6d659c5a8e69265927e71fbcf6e06bc5cc1b8ee161f
Jul 09 22:13:33.523473 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:33.523434    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk" event={"ID":"78ca84a6-3168-4db1-8425-574a9f7752fe","Type":"ContainerStarted","Data":"60b1e5fc40e122eeca8aa6d659c5a8e69265927e71fbcf6e06bc5cc1b8ee161f"}
Jul 09 22:13:34.676613 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:34.676563    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8xh4f" podUID="96332960-3e27-4437-b375-7c16a4a8a2ea"
Jul 09 22:13:35.531064 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:35.531025    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk" event={"ID":"78ca84a6-3168-4db1-8425-574a9f7752fe","Type":"ContainerStarted","Data":"7b49c09432c5aca91ea47578e73c997ef3fc30d13ee8d974f7d3780f2e665dfd"}
Jul 09 22:13:35.531310 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:35.531289    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:35.535788 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:35.535763    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk"
Jul 09 22:13:35.546115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:35.546066    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-5kfjk" podStartSLOduration=2.836596831 podStartE2EDuration="4.546048245s" totalImagesPullingTime="1.709451414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:35.545234715 +0000 UTC m=+295.436265229" watchObservedRunningTime="2026-07-09 22:13:35.546048245 +0000 UTC m=+295.437078761"
Jul 09 22:13:36.003505 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.003474    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"]
Jul 09 22:13:36.005905 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.005889    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.008960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.008928    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 22:13:36.008960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.008942    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 22:13:36.009226 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.009002    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hqjtm\""
Jul 09 22:13:36.009226 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.008966    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 22:13:36.016622 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.016592    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"]
Jul 09 22:13:36.026841 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.026810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/042a8dd6-030e-4c41-bda8-28c7e7487284-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.027001 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.026848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmlmf\" (UniqueName: \"kubernetes.io/projected/042a8dd6-030e-4c41-bda8-28c7e7487284-kube-api-access-fmlmf\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.027001 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.026926    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/042a8dd6-030e-4c41-bda8-28c7e7487284-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.027001 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.026963    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/042a8dd6-030e-4c41-bda8-28c7e7487284-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.127451 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.127414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/042a8dd6-030e-4c41-bda8-28c7e7487284-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.127655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.127460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmlmf\" (UniqueName: \"kubernetes.io/projected/042a8dd6-030e-4c41-bda8-28c7e7487284-kube-api-access-fmlmf\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.127655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.127504    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/042a8dd6-030e-4c41-bda8-28c7e7487284-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.127655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.127542    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/042a8dd6-030e-4c41-bda8-28c7e7487284-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.128298 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.128279    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/042a8dd6-030e-4c41-bda8-28c7e7487284-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.129986 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.129963    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/042a8dd6-030e-4c41-bda8-28c7e7487284-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.130075 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.130047    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/042a8dd6-030e-4c41-bda8-28c7e7487284-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.135924 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.135899    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmlmf\" (UniqueName: \"kubernetes.io/projected/042a8dd6-030e-4c41-bda8-28c7e7487284-kube-api-access-fmlmf\") pod \"prometheus-operator-567f7b8b7c-7r287\" (UID: \"042a8dd6-030e-4c41-bda8-28c7e7487284\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.315004 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.314915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"
Jul 09 22:13:36.452094 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.452060    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-7r287"]
Jul 09 22:13:36.454943 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:36.454912    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042a8dd6_030e_4c41_bda8_28c7e7487284.slice/crio-64c397a90e88c1cb4d9a8b8e5d6f94960e4f40157b947076ac898ce0d0976696 WatchSource:0}: Error finding container 64c397a90e88c1cb4d9a8b8e5d6f94960e4f40157b947076ac898ce0d0976696: Status 404 returned error can't find the container with id 64c397a90e88c1cb4d9a8b8e5d6f94960e4f40157b947076ac898ce0d0976696
Jul 09 22:13:36.535704 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:36.535665    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287" event={"ID":"042a8dd6-030e-4c41-bda8-28c7e7487284","Type":"ContainerStarted","Data":"64c397a90e88c1cb4d9a8b8e5d6f94960e4f40157b947076ac898ce0d0976696"}
Jul 09 22:13:38.544405 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:38.544369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287" event={"ID":"042a8dd6-030e-4c41-bda8-28c7e7487284","Type":"ContainerStarted","Data":"328fcff75f4c7c32df7af6408d70ee8694e319727d65cfe7f06aabe875ccc2bf"}
Jul 09 22:13:38.544804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:38.544423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287" event={"ID":"042a8dd6-030e-4c41-bda8-28c7e7487284","Type":"ContainerStarted","Data":"8513ac5509bcafef080b601cbf9608c1699f7111e8a23cd240f06f081210df56"}
Jul 09 22:13:38.560954 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:38.560906    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-7r287" podStartSLOduration=1.9622496109999998 podStartE2EDuration="3.560886265s" totalImagesPullingTime="1.598636654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:38.560299147 +0000 UTC m=+298.451329660" watchObservedRunningTime="2026-07-09 22:13:38.560886265 +0000 UTC m=+298.451916780"
Jul 09 22:13:39.513688 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.513656    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g9qfh"
Jul 09 22:13:39.650421 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.650390    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f76d99cf6-nskc2"]
Jul 09 22:13:39.652532 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.652517    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.655625 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.655603    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 22:13:39.655770 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.655601    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 22:13:39.655770 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.655647    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 22:13:39.656035 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.656016    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 22:13:39.656157 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.656057    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 22:13:39.656358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.656345    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 22:13:39.656435 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.656400    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 22:13:39.656504 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.656490    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mpx8s\""
Jul 09 22:13:39.679418 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.679389    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 22:13:39.682287 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.682229    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f76d99cf6-nskc2"]
Jul 09 22:13:39.750012 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.749978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-trusted-ca-bundle\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.750012 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.750018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-serving-cert\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.750285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.750090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-service-ca\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.750285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.750117    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-oauth-serving-cert\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.750285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.750139    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5m6m\" (UniqueName: \"kubernetes.io/projected/11d08b63-259a-44dd-a88f-6036af9d9b5c-kube-api-access-f5m6m\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.750285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.750166    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-oauth-config\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.750285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.750180    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-config\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.850761 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850657    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-trusted-ca-bundle\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.850761 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-serving-cert\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850798    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-service-ca\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850825    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-oauth-serving-cert\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850854    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5m6m\" (UniqueName: \"kubernetes.io/projected/11d08b63-259a-44dd-a88f-6036af9d9b5c-kube-api-access-f5m6m\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-oauth-config\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.850918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-config\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851692 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.851666    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-oauth-serving-cert\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851828 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.851666    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-service-ca\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851898 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.851846    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-config\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.851898 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.851851    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-trusted-ca-bundle\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.853274 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.853234    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-serving-cert\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.853498 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.853480    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-oauth-config\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.859275 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.859240    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5m6m\" (UniqueName: \"kubernetes.io/projected/11d08b63-259a-44dd-a88f-6036af9d9b5c-kube-api-access-f5m6m\") pod \"console-6f76d99cf6-nskc2\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") " pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:39.962389 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:39.962355    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:40.118872 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.118789    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f76d99cf6-nskc2"]
Jul 09 22:13:40.121705 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:40.121663    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d08b63_259a_44dd_a88f_6036af9d9b5c.slice/crio-65bce5e7ff29a2314a48921577078eec15cb320b743cb83a23f92cbe7a10ddad WatchSource:0}: Error finding container 65bce5e7ff29a2314a48921577078eec15cb320b743cb83a23f92cbe7a10ddad: Status 404 returned error can't find the container with id 65bce5e7ff29a2314a48921577078eec15cb320b743cb83a23f92cbe7a10ddad
Jul 09 22:13:40.341990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.341945    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"]
Jul 09 22:13:40.345487 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.345466    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.347608 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.347584    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 22:13:40.347729 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.347632    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f9rvs\""
Jul 09 22:13:40.347795 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.347590    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 22:13:40.354134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.353846    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hn88\" (UniqueName: \"kubernetes.io/projected/6b1d3909-2181-4287-9ad6-8e5f5949bfce-kube-api-access-2hn88\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.354134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.353916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.354134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.353963    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b1d3909-2181-4287-9ad6-8e5f5949bfce-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.354134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.353994    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.354426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.354357    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"]
Jul 09 22:13:40.380161 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.380077    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dxtjf"]
Jul 09 22:13:40.382601 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.382580    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.384864 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.384793    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 22:13:40.384864 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.384805    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 22:13:40.385074 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.384882    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 22:13:40.385168 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.385149    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mb9sv\""
Jul 09 22:13:40.454497 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454460    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-tls\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.454497 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b1d3909-2181-4287-9ad6-8e5f5949bfce-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.454716 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.454716 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-sys\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.454716 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzt9s\" (UniqueName: \"kubernetes.io/projected/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-kube-api-access-nzt9s\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.454716 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454665    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-wtmp\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.454716 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:40.454691    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 09 22:13:40.454970 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:13:40.454772    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-tls podName:6b1d3909-2181-4287-9ad6-8e5f5949bfce nodeName:}" failed. No retries permitted until 2026-07-09 22:13:40.954745403 +0000 UTC m=+300.845775908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-dmfn2" (UID: "6b1d3909-2181-4287-9ad6-8e5f5949bfce") : secret "openshift-state-metrics-tls" not found
Jul 09 22:13:40.454970 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hn88\" (UniqueName: \"kubernetes.io/projected/6b1d3909-2181-4287-9ad6-8e5f5949bfce-kube-api-access-2hn88\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.454970 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454860    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-textfile\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.454970 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454886    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.454970 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454925    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.454970 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-root\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.455266 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.454991    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-accelerators-collector-config\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.455266 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.455036    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-metrics-client-ca\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.455340 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.455296    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b1d3909-2181-4287-9ad6-8e5f5949bfce-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.457564 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.457539    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.463394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.463373    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hn88\" (UniqueName: \"kubernetes.io/projected/6b1d3909-2181-4287-9ad6-8e5f5949bfce-kube-api-access-2hn88\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.553056 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.553023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f76d99cf6-nskc2" event={"ID":"11d08b63-259a-44dd-a88f-6036af9d9b5c","Type":"ContainerStarted","Data":"65bce5e7ff29a2314a48921577078eec15cb320b743cb83a23f92cbe7a10ddad"}
Jul 09 22:13:40.555383 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555361    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-sys\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzt9s\" (UniqueName: \"kubernetes.io/projected/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-kube-api-access-nzt9s\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-wtmp\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555466    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-sys\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555522    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-textfile\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-root\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-accelerators-collector-config\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555623    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-metrics-client-ca\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-tls\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555688    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-root\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555688    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-wtmp\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.555983 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.555958    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-textfile\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.556365 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.556326    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-accelerators-collector-config\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.556483 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.556366    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-metrics-client-ca\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.563264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.558952    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-tls\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.563264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.558989    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.566068 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.566037    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzt9s\" (UniqueName: \"kubernetes.io/projected/b40b51cc-d1be-4ec8-836a-d3f9ea69c13d-kube-api-access-nzt9s\") pod \"node-exporter-dxtjf\" (UID: \"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d\") " pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.694546 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.694438    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dxtjf"
Jul 09 22:13:40.705395 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:40.705357    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40b51cc_d1be_4ec8_836a_d3f9ea69c13d.slice/crio-9849bdc6d2a136ab1f20e959ec6c5f3e98fa4dd8b346d590afd5bcf0c6b889b1 WatchSource:0}: Error finding container 9849bdc6d2a136ab1f20e959ec6c5f3e98fa4dd8b346d590afd5bcf0c6b889b1: Status 404 returned error can't find the container with id 9849bdc6d2a136ab1f20e959ec6c5f3e98fa4dd8b346d590afd5bcf0c6b889b1
Jul 09 22:13:40.958681 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.958637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:40.961892 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:40.961825    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6b1d3909-2181-4287-9ad6-8e5f5949bfce-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-dmfn2\" (UID: \"6b1d3909-2181-4287-9ad6-8e5f5949bfce\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:41.261015 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.260935    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f9rvs\""
Jul 09 22:13:41.269606 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.269577    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"
Jul 09 22:13:41.456378 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.455520    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 22:13:41.460437 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.460391    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.463535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463030    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 22:13:41.463535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463091    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 22:13:41.463535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463277    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 22:13:41.463535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463347    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 22:13:41.463535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463028    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463940    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.463992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-web-config\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464023    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464083    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464238    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-config-out\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8kwh\" (UniqueName: \"kubernetes.io/projected/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-kube-api-access-g8kwh\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464352    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.464741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464386    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.465475 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464459    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-config-volume\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.465475 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464632    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 22:13:41.465475 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.464929    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 22:13:41.465623 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.465603    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n6bpt\""
Jul 09 22:13:41.479894 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.479811    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 22:13:41.489390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.489005    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2"]
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.564929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-config-volume\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.564994    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565037    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-web-config\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565166    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565189    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565222    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-config-out\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565269    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8kwh\" (UniqueName: \"kubernetes.io/projected/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-kube-api-access-g8kwh\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.565775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.565370    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.568358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.568322    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2" event={"ID":"6b1d3909-2181-4287-9ad6-8e5f5949bfce","Type":"ContainerStarted","Data":"ee0c5b92b2dbb59b246c1cad942ef98c6d02bb97b5389c3b43a2e0ca692a81f8"}
Jul 09 22:13:41.568358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.568350    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.569424 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.569402    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-config-volume\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.569792 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.569753    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.570304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.569915    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.577660 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.577072    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxtjf" event={"ID":"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d","Type":"ContainerStarted","Data":"9849bdc6d2a136ab1f20e959ec6c5f3e98fa4dd8b346d590afd5bcf0c6b889b1"}
Jul 09 22:13:41.577660 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.577237    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.577660 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.577616    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.577926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.577865    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-config-out\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.584728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.582793    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.584728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.582815    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.584728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.583329    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-web-config\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.584728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.583647    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.585239 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.585217    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8kwh\" (UniqueName: \"kubernetes.io/projected/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-kube-api-access-g8kwh\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.585424 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.585402    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38e5ddaa-fc2d-4fb9-8250-34a822b0f73f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:41.646624 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.644886    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 22:13:41.781003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:41.780911    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 22:13:42.121128 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:42.121028    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 22:13:42.126482 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:42.126406    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e5ddaa_fc2d_4fb9_8250_34a822b0f73f.slice/crio-05488cbcb4547e8e3683a3f83629e0a5965a8a7d874fc6bd0091df64330f4551 WatchSource:0}: Error finding container 05488cbcb4547e8e3683a3f83629e0a5965a8a7d874fc6bd0091df64330f4551: Status 404 returned error can't find the container with id 05488cbcb4547e8e3683a3f83629e0a5965a8a7d874fc6bd0091df64330f4551
Jul 09 22:13:42.583512 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:42.583477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"05488cbcb4547e8e3683a3f83629e0a5965a8a7d874fc6bd0091df64330f4551"}
Jul 09 22:13:42.585926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:42.585891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2" event={"ID":"6b1d3909-2181-4287-9ad6-8e5f5949bfce","Type":"ContainerStarted","Data":"60544eeebb2e464eba2cec78ba5e9cb56e7895908970fa919b557bb1d7cdf325"}
Jul 09 22:13:42.586071 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:42.585949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2" event={"ID":"6b1d3909-2181-4287-9ad6-8e5f5949bfce","Type":"ContainerStarted","Data":"0fb1ab77bf02600fa4750e6140212dd568ea1b80c6384bc43ca444b792daabda"}
Jul 09 22:13:42.587946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:42.587916    2566 generic.go:356] "Generic (PLEG): container finished" podID="b40b51cc-d1be-4ec8-836a-d3f9ea69c13d" containerID="0206c55bdc164f60980752b91511886c6cfc28912a86dee497f7d35478a25bbb" exitCode=0
Jul 09 22:13:42.588074 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:42.587961    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxtjf" event={"ID":"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d","Type":"ContainerDied","Data":"0206c55bdc164f60980752b91511886c6cfc28912a86dee497f7d35478a25bbb"}
Jul 09 22:13:44.604426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.604388    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxtjf" event={"ID":"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d","Type":"ContainerStarted","Data":"82576cc142b704d9c59be1611e80832bd9d505ae682d65156542a1e198bca2c2"}
Jul 09 22:13:44.604861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.604485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dxtjf" event={"ID":"b40b51cc-d1be-4ec8-836a-d3f9ea69c13d","Type":"ContainerStarted","Data":"75fc2a9c8c89692cce052aeb5267da86801262a8dbd9057c2d91a8a1e6a03076"}
Jul 09 22:13:44.606065 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.606040    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f76d99cf6-nskc2" event={"ID":"11d08b63-259a-44dd-a88f-6036af9d9b5c","Type":"ContainerStarted","Data":"4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd"}
Jul 09 22:13:44.607427 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.607406    2566 generic.go:356] "Generic (PLEG): container finished" podID="38e5ddaa-fc2d-4fb9-8250-34a822b0f73f" containerID="1e0c92c0504ef8a64b6d803b5140c79878db0e92fe5e11b79b9c46c7584483f3" exitCode=0
Jul 09 22:13:44.607531 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.607480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerDied","Data":"1e0c92c0504ef8a64b6d803b5140c79878db0e92fe5e11b79b9c46c7584483f3"}
Jul 09 22:13:44.608761 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.608738    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 22:13:44.609679 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.609662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2" event={"ID":"6b1d3909-2181-4287-9ad6-8e5f5949bfce","Type":"ContainerStarted","Data":"538d5e6c79d5ffba5bec7654eb11a64b1222bea77ff17165abfbe574ba69df35"}
Jul 09 22:13:44.623482 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.623438    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dxtjf" podStartSLOduration=3.285773232 podStartE2EDuration="4.623423048s" totalImagesPullingTime="1.259527106s" totalInitContainerRuntime="78.12271ms" isStatefulPod=true podCreationTimestamp="2026-07-09 22:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:44.621990987 +0000 UTC m=+304.513021501" watchObservedRunningTime="2026-07-09 22:13:44.623423048 +0000 UTC m=+304.514453564"
Jul 09 22:13:44.638785 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.638679    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-dmfn2" podStartSLOduration=2.126833864 podStartE2EDuration="4.63865995s" totalImagesPullingTime="2.511826086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:44.638177331 +0000 UTC m=+304.529207844" watchObservedRunningTime="2026-07-09 22:13:44.63865995 +0000 UTC m=+304.529690464"
Jul 09 22:13:44.658842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:44.658791    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f76d99cf6-nskc2" podStartSLOduration=1.567645491 podStartE2EDuration="5.65877427s" totalImagesPullingTime="4.091128779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:44.657957315 +0000 UTC m=+304.548987870" watchObservedRunningTime="2026-07-09 22:13:44.65877427 +0000 UTC m=+304.549804783"
Jul 09 22:13:45.140624 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.140579    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"]
Jul 09 22:13:45.143380 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.143350    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.145655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.145603    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 09 22:13:45.145655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.145653    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kzrsz\""
Jul 09 22:13:45.152913 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.152884    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"]
Jul 09 22:13:45.201867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.201824    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72e32bdf-a214-4b93-bcb3-5f826999bc8b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2kv58\" (UID: \"72e32bdf-a214-4b93-bcb3-5f826999bc8b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.202055 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.201943    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drdd4\" (UniqueName: \"kubernetes.io/projected/72e32bdf-a214-4b93-bcb3-5f826999bc8b-kube-api-access-drdd4\") pod \"monitoring-plugin-d5dcff7cf-2kv58\" (UID: \"72e32bdf-a214-4b93-bcb3-5f826999bc8b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.303056 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.303007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drdd4\" (UniqueName: \"kubernetes.io/projected/72e32bdf-a214-4b93-bcb3-5f826999bc8b-kube-api-access-drdd4\") pod \"monitoring-plugin-d5dcff7cf-2kv58\" (UID: \"72e32bdf-a214-4b93-bcb3-5f826999bc8b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.303293 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.303091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72e32bdf-a214-4b93-bcb3-5f826999bc8b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2kv58\" (UID: \"72e32bdf-a214-4b93-bcb3-5f826999bc8b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.306701 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.306620    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72e32bdf-a214-4b93-bcb3-5f826999bc8b-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-2kv58\" (UID: \"72e32bdf-a214-4b93-bcb3-5f826999bc8b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.313069 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.313039    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drdd4\" (UniqueName: \"kubernetes.io/projected/72e32bdf-a214-4b93-bcb3-5f826999bc8b-kube-api-access-drdd4\") pod \"monitoring-plugin-d5dcff7cf-2kv58\" (UID: \"72e32bdf-a214-4b93-bcb3-5f826999bc8b\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.457998 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.457964    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:45.600549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.600524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"]
Jul 09 22:13:45.602818 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:13:45.602792    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e32bdf_a214_4b93_bcb3_5f826999bc8b.slice/crio-7b81ff8a32802229ce9b32b7457429a018752095d5c3b9d482c6b3a37f8928b2 WatchSource:0}: Error finding container 7b81ff8a32802229ce9b32b7457429a018752095d5c3b9d482c6b3a37f8928b2: Status 404 returned error can't find the container with id 7b81ff8a32802229ce9b32b7457429a018752095d5c3b9d482c6b3a37f8928b2
Jul 09 22:13:45.615005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.614958    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58" event={"ID":"72e32bdf-a214-4b93-bcb3-5f826999bc8b","Type":"ContainerStarted","Data":"7b81ff8a32802229ce9b32b7457429a018752095d5c3b9d482c6b3a37f8928b2"}
Jul 09 22:13:45.674084 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:45.674053    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:13:46.086224 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.086186    2566 patch_prober.go:28] interesting pod/image-registry-5cc865548f-9rtph container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 22:13:46.086426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.086274    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 22:13:46.237113 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.237078    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cc865548f-9rtph"]
Jul 09 22:13:46.242378 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.242344    2566 patch_prober.go:28] interesting pod/image-registry-5cc865548f-9rtph container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 22:13:46.242540 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.242400    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 22:13:46.632613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.632370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"2c14e1ffd08fb3e44211de59c710ef597e07558afc164050687cb1e61f75de0e"}
Jul 09 22:13:46.632613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.632494    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"1a21a667d6784d40872ad1da939f24d6c53e5a06648a851b2ac99b3063dbfd42"}
Jul 09 22:13:46.632613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:46.632547    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"dd33b2f49fb4658f59e4a2f14513254ab842dbaa36b0701a6abf7cad6bdc90cf"}
Jul 09 22:13:47.640375 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:47.640322    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"12ad43ce30ae0f530c289f7a4361a2ef34abe8e2942aa939b30f4e292616d5ae"}
Jul 09 22:13:47.640749 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:47.640390    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"df3ab695b088d3e30aa7e3413bedae61f0a60e639e20a46729ffe3461fa0db3b"}
Jul 09 22:13:48.646013 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:48.645967    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58" event={"ID":"72e32bdf-a214-4b93-bcb3-5f826999bc8b","Type":"ContainerStarted","Data":"e5ac1887a5dda99463625b9a9acfc94903726a679d20ce39bd967f5c5d00df47"}
Jul 09 22:13:48.646511 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:48.646110    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:48.649798 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:48.649767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38e5ddaa-fc2d-4fb9-8250-34a822b0f73f","Type":"ContainerStarted","Data":"904589f5a6dfc8a6ff515e503ab91d241cce94fcc61d153ce20895547dfd8887"}
Jul 09 22:13:48.651424 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:48.651400    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58"
Jul 09 22:13:48.663369 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:48.663328    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-2kv58" podStartSLOduration=1.59904008 podStartE2EDuration="3.663313391s" totalImagesPullingTime="2.064273311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:48.661896135 +0000 UTC m=+308.552926827" watchObservedRunningTime="2026-07-09 22:13:48.663313391 +0000 UTC m=+308.554343914"
Jul 09 22:13:48.717128 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:48.716945    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.915233804 podStartE2EDuration="7.716928439s" totalImagesPullingTime="4.780604722s" totalInitContainerRuntime="21.089913ms" isStatefulPod=false podCreationTimestamp="2026-07-09 22:13:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:13:48.714582021 +0000 UTC m=+308.605612558" watchObservedRunningTime="2026-07-09 22:13:48.716928439 +0000 UTC m=+308.607958953"
Jul 09 22:13:49.963189 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:49.963142    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:49.963609 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:49.963211    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:49.968077 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:49.968053    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:50.662742 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:50.662715    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:13:55.966744 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:55.966702    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f76d99cf6-nskc2"]
Jul 09 22:13:56.241740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:13:56.241667    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:14:10.739165 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:10.739131    2566 generic.go:356] "Generic (PLEG): container finished" podID="11d804de-7aa9-4a6b-a90d-a9b10ef6f463" containerID="d45d3e2f32bb9bb258ab02d361938b8d8eb824420e8568617a7a348c1c4053bf" exitCode=0
Jul 09 22:14:10.739625 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:10.739206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2" event={"ID":"11d804de-7aa9-4a6b-a90d-a9b10ef6f463","Type":"ContainerDied","Data":"d45d3e2f32bb9bb258ab02d361938b8d8eb824420e8568617a7a348c1c4053bf"}
Jul 09 22:14:10.739625 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:10.739537    2566 scope.go:117] "RemoveContainer" containerID="d45d3e2f32bb9bb258ab02d361938b8d8eb824420e8568617a7a348c1c4053bf"
Jul 09 22:14:11.261136 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.261081    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerName="registry" containerID="cri-o://b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d" gracePeriod=30
Jul 09 22:14:11.517564 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.517503    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:14:11.645754 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645714    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-certificates\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.645926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645771    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-installation-pull-secrets\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.645926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645821    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-ca-trust-extracted\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.645926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645839    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.645926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645891    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-bound-sa-token\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.645926 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645915    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pxp9\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-kube-api-access-5pxp9\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.646813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.645959    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-trusted-ca\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.646813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.646086    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-image-registry-private-configuration\") pod \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\" (UID: \"ed48d949-2eb1-43bd-8c4f-e2e163a299ff\") "
Jul 09 22:14:11.646813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.646207    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:14:11.646813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.646472    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-certificates\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.647115 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.647087    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:14:11.649063 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.649010    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:14:11.649170 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.649076    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:14:11.649170 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.649092    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:14:11.649170 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.649125    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:14:11.649360 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.649218    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-kube-api-access-5pxp9" (OuterVolumeSpecName: "kube-api-access-5pxp9") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "kube-api-access-5pxp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:14:11.655675 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.655648    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ed48d949-2eb1-43bd-8c4f-e2e163a299ff" (UID: "ed48d949-2eb1-43bd-8c4f-e2e163a299ff"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:14:11.745713 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.745678    2566 generic.go:356] "Generic (PLEG): container finished" podID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerID="b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d" exitCode=0
Jul 09 22:14:11.746111 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.745767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" event={"ID":"ed48d949-2eb1-43bd-8c4f-e2e163a299ff","Type":"ContainerDied","Data":"b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d"}
Jul 09 22:14:11.746111 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.745819    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cc865548f-9rtph" event={"ID":"ed48d949-2eb1-43bd-8c4f-e2e163a299ff","Type":"ContainerDied","Data":"a94337efbc3298f51d94682041e1d20eda52842748e3ba4cf885dcb49b98222a"}
Jul 09 22:14:11.746111 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.745845    2566 scope.go:117] "RemoveContainer" containerID="b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d"
Jul 09 22:14:11.746111 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.745988    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cc865548f-9rtph"
Jul 09 22:14:11.746966 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.746941    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-ca-trust-extracted\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.747098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.746969    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-registry-tls\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.747098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.746983    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-bound-sa-token\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.747098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.746996    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pxp9\" (UniqueName: \"kubernetes.io/projected/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-kube-api-access-5pxp9\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.747098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.747010    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-trusted-ca\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.747098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.747023    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-image-registry-private-configuration\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.747098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.747038    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ed48d949-2eb1-43bd-8c4f-e2e163a299ff-installation-pull-secrets\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:11.748843 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.748809    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-br6d2" event={"ID":"11d804de-7aa9-4a6b-a90d-a9b10ef6f463","Type":"ContainerStarted","Data":"b6a67da6ff2255d4710ec054369167616ba3fcef29794a5cd34b506d7fa85304"}
Jul 09 22:14:11.755463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.755444    2566 scope.go:117] "RemoveContainer" containerID="b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d"
Jul 09 22:14:11.755731 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:14:11.755710    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d\": container with ID starting with b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d not found: ID does not exist" containerID="b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d"
Jul 09 22:14:11.755800 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.755744    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d"} err="failed to get container status \"b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d\": rpc error: code = NotFound desc = could not find container \"b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d\": container with ID starting with b6008faab67e4b1292dca0cb73f6eb85c6d546b5346c3da731cd233c2b61c58d not found: ID does not exist"
Jul 09 22:14:11.788371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.788287    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cc865548f-9rtph"]
Jul 09 22:14:11.796095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:11.796062    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cc865548f-9rtph"]
Jul 09 22:14:13.678840 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:13.678807    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" path="/var/lib/kubelet/pods/ed48d949-2eb1-43bd-8c4f-e2e163a299ff/volumes"
Jul 09 22:14:20.987682 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:20.987634    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6f76d99cf6-nskc2" podUID="11d08b63-259a-44dd-a88f-6036af9d9b5c" containerName="console" containerID="cri-o://4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd" gracePeriod=15
Jul 09 22:14:21.265843 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.265818    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:14:21.334436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334403    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5m6m\" (UniqueName: \"kubernetes.io/projected/11d08b63-259a-44dd-a88f-6036af9d9b5c-kube-api-access-f5m6m\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.334640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334456    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-oauth-serving-cert\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.334640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334487    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-service-ca\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.334640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334526    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-oauth-config\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.334640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334554    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-serving-cert\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.334640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334575    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-config\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.334640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.334605    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-trusted-ca-bundle\") pod \"11d08b63-259a-44dd-a88f-6036af9d9b5c\" (UID: \"11d08b63-259a-44dd-a88f-6036af9d9b5c\") "
Jul 09 22:14:21.335094 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.335030    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:14:21.335192 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.335091    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-service-ca" (OuterVolumeSpecName: "service-ca") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:14:21.335192 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.335091    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-config" (OuterVolumeSpecName: "console-config") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:14:21.335192 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.335176    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:14:21.336806 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.336784    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:14:21.337095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.337075    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d08b63-259a-44dd-a88f-6036af9d9b5c-kube-api-access-f5m6m" (OuterVolumeSpecName: "kube-api-access-f5m6m") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "kube-api-access-f5m6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:14:21.337146 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.337085    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11d08b63-259a-44dd-a88f-6036af9d9b5c" (UID: "11d08b63-259a-44dd-a88f-6036af9d9b5c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:14:21.435661 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435619    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-trusted-ca-bundle\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.435661 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435653    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5m6m\" (UniqueName: \"kubernetes.io/projected/11d08b63-259a-44dd-a88f-6036af9d9b5c-kube-api-access-f5m6m\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.435661 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435667    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-oauth-serving-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.435896 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435680    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-service-ca\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.435896 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435692    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-oauth-config\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.435896 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435704    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-serving-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.435896 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.435715    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11d08b63-259a-44dd-a88f-6036af9d9b5c-console-config\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:14:21.789547 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.789511    2566 generic.go:356] "Generic (PLEG): container finished" podID="ecd5587b-22e2-4d24-a201-e37a074f83ad" containerID="3a9e892cf89e35a42a1dffb069f6cf1a8d5aa2988301943a1d33deb366e51d5d" exitCode=0
Jul 09 22:14:21.789715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.789587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s" event={"ID":"ecd5587b-22e2-4d24-a201-e37a074f83ad","Type":"ContainerDied","Data":"3a9e892cf89e35a42a1dffb069f6cf1a8d5aa2988301943a1d33deb366e51d5d"}
Jul 09 22:14:21.789983 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.789947    2566 scope.go:117] "RemoveContainer" containerID="3a9e892cf89e35a42a1dffb069f6cf1a8d5aa2988301943a1d33deb366e51d5d"
Jul 09 22:14:21.791050 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.791026    2566 generic.go:356] "Generic (PLEG): container finished" podID="11d08b63-259a-44dd-a88f-6036af9d9b5c" containerID="4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd" exitCode=0
Jul 09 22:14:21.791137 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.791083    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f76d99cf6-nskc2"
Jul 09 22:14:21.791137 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.791095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f76d99cf6-nskc2" event={"ID":"11d08b63-259a-44dd-a88f-6036af9d9b5c","Type":"ContainerDied","Data":"4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd"}
Jul 09 22:14:21.791222 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.791137    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f76d99cf6-nskc2" event={"ID":"11d08b63-259a-44dd-a88f-6036af9d9b5c","Type":"ContainerDied","Data":"65bce5e7ff29a2314a48921577078eec15cb320b743cb83a23f92cbe7a10ddad"}
Jul 09 22:14:21.791222 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.791164    2566 scope.go:117] "RemoveContainer" containerID="4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd"
Jul 09 22:14:21.801405 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.801389    2566 scope.go:117] "RemoveContainer" containerID="4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd"
Jul 09 22:14:21.801696 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:14:21.801678    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd\": container with ID starting with 4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd not found: ID does not exist" containerID="4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd"
Jul 09 22:14:21.801747 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.801705    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd"} err="failed to get container status \"4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd\": rpc error: code = NotFound desc = could not find container \"4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd\": container with ID starting with 4909cb0a821540e14633fd8ba74e9bf5fd7089a3250f5bc8558913f043f7ccbd not found: ID does not exist"
Jul 09 22:14:21.821183 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.821155    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f76d99cf6-nskc2"]
Jul 09 22:14:21.824838 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:21.824813    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f76d99cf6-nskc2"]
Jul 09 22:14:22.798919 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:22.798882    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mbc8s" event={"ID":"ecd5587b-22e2-4d24-a201-e37a074f83ad","Type":"ContainerStarted","Data":"60fe7e1a02bcddfadf6973c37ba7df0e72ae07db29fab529580c172f95cd7f0f"}
Jul 09 22:14:23.679204 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:23.679169    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d08b63-259a-44dd-a88f-6036af9d9b5c" path="/var/lib/kubelet/pods/11d08b63-259a-44dd-a88f-6036af9d9b5c/volumes"
Jul 09 22:14:31.835611 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:31.835575    2566 generic.go:356] "Generic (PLEG): container finished" podID="edc94141-8082-4394-a322-f66937bd7fb1" containerID="52c1d17cfdcf9a8a57f1c4f4910ea823e159c98e8aadc4612e1843f9f4b669eb" exitCode=0
Jul 09 22:14:31.835990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:31.835638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j" event={"ID":"edc94141-8082-4394-a322-f66937bd7fb1","Type":"ContainerDied","Data":"52c1d17cfdcf9a8a57f1c4f4910ea823e159c98e8aadc4612e1843f9f4b669eb"}
Jul 09 22:14:31.835990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:31.835913    2566 scope.go:117] "RemoveContainer" containerID="52c1d17cfdcf9a8a57f1c4f4910ea823e159c98e8aadc4612e1843f9f4b669eb"
Jul 09 22:14:32.842667 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:32.842629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-9s76j" event={"ID":"edc94141-8082-4394-a322-f66937bd7fb1","Type":"ContainerStarted","Data":"c7c9a78320ca2733ea916abb284b6ed3772302b00638abb1df2958b5117bbaf7"}
Jul 09 22:14:55.140728 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:55.140688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:14:55.142943 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:55.142922    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96332960-3e27-4437-b375-7c16a4a8a2ea-metrics-certs\") pod \"network-metrics-daemon-8xh4f\" (UID: \"96332960-3e27-4437-b375-7c16a4a8a2ea\") " pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:14:55.277781 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:55.277745    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zm9fl\""
Jul 09 22:14:55.285842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:55.285816    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8xh4f"
Jul 09 22:14:55.419628 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:55.419483    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8xh4f"]
Jul 09 22:14:55.422223 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:14:55.422197    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96332960_3e27_4437_b375_7c16a4a8a2ea.slice/crio-470bc39a3a29c38d350ee4a7bdb90ffd1c50acfe0f998efc4f462ad96a9ac71a WatchSource:0}: Error finding container 470bc39a3a29c38d350ee4a7bdb90ffd1c50acfe0f998efc4f462ad96a9ac71a: Status 404 returned error can't find the container with id 470bc39a3a29c38d350ee4a7bdb90ffd1c50acfe0f998efc4f462ad96a9ac71a
Jul 09 22:14:55.937291 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:55.937233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8xh4f" event={"ID":"96332960-3e27-4437-b375-7c16a4a8a2ea","Type":"ContainerStarted","Data":"470bc39a3a29c38d350ee4a7bdb90ffd1c50acfe0f998efc4f462ad96a9ac71a"}
Jul 09 22:14:56.942700 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:56.942604    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8xh4f" event={"ID":"96332960-3e27-4437-b375-7c16a4a8a2ea","Type":"ContainerStarted","Data":"991547742f249221ee01858fd167ee89b85fd90c820793e77cd1417a099d2e28"}
Jul 09 22:14:56.942700 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:56.942671    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8xh4f" event={"ID":"96332960-3e27-4437-b375-7c16a4a8a2ea","Type":"ContainerStarted","Data":"b417d9be3ffb479746962bd3bbf350de03d51d81038571f228b20881f3fc5ade"}
Jul 09 22:14:56.958618 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:14:56.958565    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8xh4f" podStartSLOduration=373.78035359 podStartE2EDuration="6m14.95855008s" totalImagesPullingTime="1.17819649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:14:56.956656559 +0000 UTC m=+376.847687072" watchObservedRunningTime="2026-07-09 22:14:56.95855008 +0000 UTC m=+376.849580595"
Jul 09 22:15:04.696825 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.696774    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69b889f67-whbfj"]
Jul 09 22:15:04.697824 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.697801    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerName="registry"
Jul 09 22:15:04.697953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.697942    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerName="registry"
Jul 09 22:15:04.698050 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.698041    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d08b63-259a-44dd-a88f-6036af9d9b5c" containerName="console"
Jul 09 22:15:04.698122 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.698113    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d08b63-259a-44dd-a88f-6036af9d9b5c" containerName="console"
Jul 09 22:15:04.702426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.702323    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed48d949-2eb1-43bd-8c4f-e2e163a299ff" containerName="registry"
Jul 09 22:15:04.702570 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.702447    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d08b63-259a-44dd-a88f-6036af9d9b5c" containerName="console"
Jul 09 22:15:04.708626 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.708596    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.711590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.711560    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 22:15:04.711758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.711618    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 22:15:04.711758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.711650    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 22:15:04.711758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.711562    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 22:15:04.712067 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.711798    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 22:15:04.712067 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.712029    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-x7cqp\""
Jul 09 22:15:04.712210 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.712155    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b889f67-whbfj"]
Jul 09 22:15:04.716862 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.716838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-telemeter-client-tls\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717011 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.716880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717011 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.716942    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-secret-telemeter-client\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717011 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.716982    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-federate-client-tls\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717011 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.717005    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnj5f\" (UniqueName: \"kubernetes.io/projected/3dcae647-902f-41eb-854d-f49604d5ccde-kube-api-access-qnj5f\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717286 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.717033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-serving-certs-ca-bundle\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717286 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.717056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.717286 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.717103    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-metrics-client-ca\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.718086 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.718064    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 22:15:04.818171 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818129    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-telemeter-client-tls\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818171 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818205    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-secret-telemeter-client\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-federate-client-tls\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnj5f\" (UniqueName: \"kubernetes.io/projected/3dcae647-902f-41eb-854d-f49604d5ccde-kube-api-access-qnj5f\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-serving-certs-ca-bundle\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818302    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.818480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.818329    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-metrics-client-ca\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.819208 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.819181    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-serving-certs-ca-bundle\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.819405 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.819205    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.819505 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.819224    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dcae647-902f-41eb-854d-f49604d5ccde-metrics-client-ca\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.820982 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.820954    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-secret-telemeter-client\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.820982 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.820971    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-telemeter-client-tls\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.821201 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.821180    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.821279 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.821205    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3dcae647-902f-41eb-854d-f49604d5ccde-federate-client-tls\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:04.826104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:04.826078    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnj5f\" (UniqueName: \"kubernetes.io/projected/3dcae647-902f-41eb-854d-f49604d5ccde-kube-api-access-qnj5f\") pod \"telemeter-client-69b889f67-whbfj\" (UID: \"3dcae647-902f-41eb-854d-f49604d5ccde\") " pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:05.022039 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:05.021993    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj"
Jul 09 22:15:05.178808 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:05.178769    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69b889f67-whbfj"]
Jul 09 22:15:05.182230 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:15:05.182201    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dcae647_902f_41eb_854d_f49604d5ccde.slice/crio-fa5fb3b04c1c9fed53d1303b7ae7948d60a3df3eb044a9a1914de55e89755cd5 WatchSource:0}: Error finding container fa5fb3b04c1c9fed53d1303b7ae7948d60a3df3eb044a9a1914de55e89755cd5: Status 404 returned error can't find the container with id fa5fb3b04c1c9fed53d1303b7ae7948d60a3df3eb044a9a1914de55e89755cd5
Jul 09 22:15:05.980724 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:05.980691    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj" event={"ID":"3dcae647-902f-41eb-854d-f49604d5ccde","Type":"ContainerStarted","Data":"fa5fb3b04c1c9fed53d1303b7ae7948d60a3df3eb044a9a1914de55e89755cd5"}
Jul 09 22:15:09.001457 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:09.001418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj" event={"ID":"3dcae647-902f-41eb-854d-f49604d5ccde","Type":"ContainerStarted","Data":"7a61e24a20ae59fc831d17a668d0d802603910ad6b8572dc54fe7136ac80e3c3"}
Jul 09 22:15:09.001868 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:09.001524    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj" event={"ID":"3dcae647-902f-41eb-854d-f49604d5ccde","Type":"ContainerStarted","Data":"505a5b93100c31a66ef275669ec3c11f7ebe82ac5d5b316a7300902445deb0f8"}
Jul 09 22:15:09.001868 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:09.001554    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj" event={"ID":"3dcae647-902f-41eb-854d-f49604d5ccde","Type":"ContainerStarted","Data":"f76c66728933cfc60c36f3699f9787155aca12c0270a167c6e2b37e734a34a51"}
Jul 09 22:15:09.023789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:09.023719    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69b889f67-whbfj" podStartSLOduration=1.93311598 podStartE2EDuration="5.023699689s" totalImagesPullingTime="3.090583709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:09.021544096 +0000 UTC m=+388.912574611" watchObservedRunningTime="2026-07-09 22:15:09.023699689 +0000 UTC m=+388.914730204"
Jul 09 22:15:10.324415 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.324382    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79d6878989-szksj"]
Jul 09 22:15:10.327812 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.327795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.330746 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330721    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mpx8s\""
Jul 09 22:15:10.330746 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330731    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 22:15:10.330946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330804    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 22:15:10.330946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330820    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 22:15:10.330946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330830    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 22:15:10.330946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330842    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 22:15:10.331150 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.330983    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 22:15:10.331224 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.331209    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 22:15:10.335640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.335619    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 22:15:10.338626 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.338604    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79d6878989-szksj"]
Jul 09 22:15:10.361220 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361193    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-service-ca\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.361371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-oauth-config\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.361371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361288    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-oauth-serving-cert\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.361371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-trusted-ca-bundle\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.361371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361355    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fgt2\" (UniqueName: \"kubernetes.io/projected/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-kube-api-access-7fgt2\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.361501 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-config\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.361501 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.361423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-serving-cert\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462197 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-serving-cert\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462210    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-service-ca\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-oauth-config\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462292    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-oauth-serving-cert\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462321    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-trusted-ca-bundle\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462370    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fgt2\" (UniqueName: \"kubernetes.io/projected/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-kube-api-access-7fgt2\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.462423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.462406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-config\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.463134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.463107    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-service-ca\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.463285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.463170    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-config\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.463285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.463192    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-oauth-serving-cert\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.463506 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.463483    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-trusted-ca-bundle\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.464829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.464809    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-serving-cert\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.464958 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.464940    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-oauth-config\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.470528 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.470508    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fgt2\" (UniqueName: \"kubernetes.io/projected/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-kube-api-access-7fgt2\") pod \"console-79d6878989-szksj\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") " pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.638303 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.638219    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:10.770563 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:10.770529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79d6878989-szksj"]
Jul 09 22:15:10.773566 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:15:10.773529    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ab7b0e_2e9a_4695_9b39_861c2bf34d93.slice/crio-eeaa2df227d0558309f21bc53d0afddcb814a245665899d02dcd5acf3e8ffa53 WatchSource:0}: Error finding container eeaa2df227d0558309f21bc53d0afddcb814a245665899d02dcd5acf3e8ffa53: Status 404 returned error can't find the container with id eeaa2df227d0558309f21bc53d0afddcb814a245665899d02dcd5acf3e8ffa53
Jul 09 22:15:11.011891 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:11.011856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6878989-szksj" event={"ID":"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93","Type":"ContainerStarted","Data":"c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6"}
Jul 09 22:15:11.012065 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:11.011903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6878989-szksj" event={"ID":"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93","Type":"ContainerStarted","Data":"eeaa2df227d0558309f21bc53d0afddcb814a245665899d02dcd5acf3e8ffa53"}
Jul 09 22:15:11.028925 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:11.028880    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79d6878989-szksj" podStartSLOduration=1.028865552 podStartE2EDuration="1.028865552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:11.027362863 +0000 UTC m=+390.918393378" watchObservedRunningTime="2026-07-09 22:15:11.028865552 +0000 UTC m=+390.919896067"
Jul 09 22:15:19.806483 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.806448    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79d6878989-szksj"]
Jul 09 22:15:19.837633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.837593    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-659897f855-tc4kk"]
Jul 09 22:15:19.842699 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.842676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.850838 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.850689    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659897f855-tc4kk"]
Jul 09 22:15:19.951833 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.951788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-service-ca\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.951833 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.951835    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-serving-cert\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.952104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.951870    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-oauth-serving-cert\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.952104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.951894    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-config\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.952104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.951971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-oauth-config\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.952104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.952022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-trusted-ca-bundle\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:19.952104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:19.952072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qllmr\" (UniqueName: \"kubernetes.io/projected/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-kube-api-access-qllmr\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053468    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qllmr\" (UniqueName: \"kubernetes.io/projected/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-kube-api-access-qllmr\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053692 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053613    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-service-ca\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053692 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-serving-cert\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053779 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-oauth-serving-cert\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053779 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053728    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-config\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053779 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-oauth-config\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.053779 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.053772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-trusted-ca-bundle\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.054425 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.054399    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-service-ca\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.054624 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.054481    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-config\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.054624 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.054486    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-oauth-serving-cert\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.054877 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.054856    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-trusted-ca-bundle\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.056396 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.056373    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-oauth-config\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.056615 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.056571    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-serving-cert\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.064945 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.064923    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qllmr\" (UniqueName: \"kubernetes.io/projected/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-kube-api-access-qllmr\") pod \"console-659897f855-tc4kk\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") " pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.154024 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.153986    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:20.287309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.287159    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659897f855-tc4kk"]
Jul 09 22:15:20.290220 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:15:20.290194    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e042d50_5e80_4a5e_96dc_6f5b842f5a07.slice/crio-9d95fdc49b09fda2eece24032d9ef3210f74b059beb8cd12e4bca1b4b49e95ba WatchSource:0}: Error finding container 9d95fdc49b09fda2eece24032d9ef3210f74b059beb8cd12e4bca1b4b49e95ba: Status 404 returned error can't find the container with id 9d95fdc49b09fda2eece24032d9ef3210f74b059beb8cd12e4bca1b4b49e95ba
Jul 09 22:15:20.639176 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:20.639076    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:21.053302 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:21.053263    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659897f855-tc4kk" event={"ID":"3e042d50-5e80-4a5e-96dc-6f5b842f5a07","Type":"ContainerStarted","Data":"329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c"}
Jul 09 22:15:21.053692 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:21.053316    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659897f855-tc4kk" event={"ID":"3e042d50-5e80-4a5e-96dc-6f5b842f5a07","Type":"ContainerStarted","Data":"9d95fdc49b09fda2eece24032d9ef3210f74b059beb8cd12e4bca1b4b49e95ba"}
Jul 09 22:15:21.070305 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:21.070221    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-659897f855-tc4kk" podStartSLOduration=2.070201072 podStartE2EDuration="2.070201072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:15:21.068905006 +0000 UTC m=+400.959935522" watchObservedRunningTime="2026-07-09 22:15:21.070201072 +0000 UTC m=+400.961231585"
Jul 09 22:15:30.155109 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:30.155070    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:30.155642 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:30.155176    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:30.160085 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:30.160067    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:31.097125 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:31.097093    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:15:44.827457 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:44.827397    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-79d6878989-szksj" podUID="e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" containerName="console" containerID="cri-o://c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6" gracePeriod=15
Jul 09 22:15:45.079156 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.079087    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:45.155795 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.155760    2566 generic.go:356] "Generic (PLEG): container finished" podID="e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" containerID="c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6" exitCode=0
Jul 09 22:15:45.155795 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.155799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6878989-szksj" event={"ID":"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93","Type":"ContainerDied","Data":"c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6"}
Jul 09 22:15:45.156005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.155809    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79d6878989-szksj"
Jul 09 22:15:45.156005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.155840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79d6878989-szksj" event={"ID":"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93","Type":"ContainerDied","Data":"eeaa2df227d0558309f21bc53d0afddcb814a245665899d02dcd5acf3e8ffa53"}
Jul 09 22:15:45.156005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.155859    2566 scope.go:117] "RemoveContainer" containerID="c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6"
Jul 09 22:15:45.164138 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.164121    2566 scope.go:117] "RemoveContainer" containerID="c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6"
Jul 09 22:15:45.164445 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:15:45.164424    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6\": container with ID starting with c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6 not found: ID does not exist" containerID="c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6"
Jul 09 22:15:45.164502 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.164456    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6"} err="failed to get container status \"c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6\": rpc error: code = NotFound desc = could not find container \"c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6\": container with ID starting with c22b0a78f244906e3e4e4ffd7119607b2d4995f2a044de71960858b20c39e5d6 not found: ID does not exist"
Jul 09 22:15:45.179737 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.179715    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-oauth-serving-cert\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.179812 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.179752    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fgt2\" (UniqueName: \"kubernetes.io/projected/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-kube-api-access-7fgt2\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.179812 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.179795    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-serving-cert\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.179929 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.179902    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-service-ca\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.180014 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.179997    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-trusted-ca-bundle\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.180128 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180033    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-config\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.180128 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180076    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-oauth-config\") pod \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\" (UID: \"e3ab7b0e-2e9a-4695-9b39-861c2bf34d93\") "
Jul 09 22:15:45.180230 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180187    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:15:45.180429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180404    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-service-ca" (OuterVolumeSpecName: "service-ca") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:15:45.180504 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180459    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:15:45.180504 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180453    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-config" (OuterVolumeSpecName: "console-config") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:15:45.180605 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180520    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-trusted-ca-bundle\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.180605 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180532    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-config\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.180605 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180541    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-oauth-serving-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.180605 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.180554    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-service-ca\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.182009 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.181988    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:15:45.182082 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.182055    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-kube-api-access-7fgt2" (OuterVolumeSpecName: "kube-api-access-7fgt2") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "kube-api-access-7fgt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:15:45.182141 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.182114    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" (UID: "e3ab7b0e-2e9a-4695-9b39-861c2bf34d93"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:15:45.281753 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.281707    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-oauth-config\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.281753 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.281746    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fgt2\" (UniqueName: \"kubernetes.io/projected/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-kube-api-access-7fgt2\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.281753 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.281761    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93-console-serving-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:15:45.476628 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.476599    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79d6878989-szksj"]
Jul 09 22:15:45.478616 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.478592    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79d6878989-szksj"]
Jul 09 22:15:45.679073 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:15:45.679035    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" path="/var/lib/kubelet/pods/e3ab7b0e-2e9a-4695-9b39-861c2bf34d93/volumes"
Jul 09 22:16:51.602158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.602122    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"]
Jul 09 22:16:51.602707 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.602642    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" containerName="console"
Jul 09 22:16:51.602707 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.602663    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" containerName="console"
Jul 09 22:16:51.602823 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.602767    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3ab7b0e-2e9a-4695-9b39-861c2bf34d93" containerName="console"
Jul 09 22:16:51.605987 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.605966    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.610536 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.610512    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hbvf5\""
Jul 09 22:16:51.610666 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.610542    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 22:16:51.611261 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.611231    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 22:16:51.614907 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.614887    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"]
Jul 09 22:16:51.638401 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.638376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xf6zn\" (UID: \"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.638513 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.638406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7shgv\" (UniqueName: \"kubernetes.io/projected/4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4-kube-api-access-7shgv\") pod \"cert-manager-cainjector-559cdd8c87-xf6zn\" (UID: \"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.739702 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.739670    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xf6zn\" (UID: \"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.739702 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.739707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7shgv\" (UniqueName: \"kubernetes.io/projected/4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4-kube-api-access-7shgv\") pod \"cert-manager-cainjector-559cdd8c87-xf6zn\" (UID: \"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.749899 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.749865    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xf6zn\" (UID: \"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.750017 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.749941    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7shgv\" (UniqueName: \"kubernetes.io/projected/4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4-kube-api-access-7shgv\") pod \"cert-manager-cainjector-559cdd8c87-xf6zn\" (UID: \"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:51.926420 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:51.926339    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"
Jul 09 22:16:52.053814 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:52.053786    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn"]
Jul 09 22:16:52.056528 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:16:52.056501    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d1e526b_e4e9_4b03_8eb7_3dd28ff501d4.slice/crio-349ad6cce48428596abab42bb740afa0ccc560eabf9203cda32c8ec237ccb6cd WatchSource:0}: Error finding container 349ad6cce48428596abab42bb740afa0ccc560eabf9203cda32c8ec237ccb6cd: Status 404 returned error can't find the container with id 349ad6cce48428596abab42bb740afa0ccc560eabf9203cda32c8ec237ccb6cd
Jul 09 22:16:52.445632 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:52.445595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn" event={"ID":"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4","Type":"ContainerStarted","Data":"349ad6cce48428596abab42bb740afa0ccc560eabf9203cda32c8ec237ccb6cd"}
Jul 09 22:16:55.460412 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:55.460380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn" event={"ID":"4d1e526b-e4e9-4b03-8eb7-3dd28ff501d4","Type":"ContainerStarted","Data":"a0cfa8844b1af0ff195c0e57d7e9fad7a9f87b36875c4929c19f16e390e4d0ff"}
Jul 09 22:16:55.476304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:55.476240    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xf6zn" podStartSLOduration=1.465928608 podStartE2EDuration="4.476222607s" totalImagesPullingTime="3.010293999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:16:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:16:55.474746061 +0000 UTC m=+495.365776586" watchObservedRunningTime="2026-07-09 22:16:55.476222607 +0000 UTC m=+495.367253120"
Jul 09 22:16:57.200193 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.200156    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ctrxp"]
Jul 09 22:16:57.203354 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.203333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.205874 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.205854    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-grmfg\""
Jul 09 22:16:57.211584 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.211561    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ctrxp"]
Jul 09 22:16:57.290375 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.290336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/969cc86e-0ec0-42e1-affb-539139f620e0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ctrxp\" (UID: \"969cc86e-0ec0-42e1-affb-539139f620e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.290560 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.290384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcvlk\" (UniqueName: \"kubernetes.io/projected/969cc86e-0ec0-42e1-affb-539139f620e0-kube-api-access-zcvlk\") pod \"cert-manager-webhook-6486ffd997-ctrxp\" (UID: \"969cc86e-0ec0-42e1-affb-539139f620e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.391400 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.391365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/969cc86e-0ec0-42e1-affb-539139f620e0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ctrxp\" (UID: \"969cc86e-0ec0-42e1-affb-539139f620e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.391557 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.391415    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcvlk\" (UniqueName: \"kubernetes.io/projected/969cc86e-0ec0-42e1-affb-539139f620e0-kube-api-access-zcvlk\") pod \"cert-manager-webhook-6486ffd997-ctrxp\" (UID: \"969cc86e-0ec0-42e1-affb-539139f620e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.402155 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.402120    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/969cc86e-0ec0-42e1-affb-539139f620e0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-ctrxp\" (UID: \"969cc86e-0ec0-42e1-affb-539139f620e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.402285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.402179    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcvlk\" (UniqueName: \"kubernetes.io/projected/969cc86e-0ec0-42e1-affb-539139f620e0-kube-api-access-zcvlk\") pod \"cert-manager-webhook-6486ffd997-ctrxp\" (UID: \"969cc86e-0ec0-42e1-affb-539139f620e0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.512478 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.512423    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:57.665667 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:57.665635    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-ctrxp"]
Jul 09 22:16:57.669127 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:16:57.669086    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod969cc86e_0ec0_42e1_affb_539139f620e0.slice/crio-3b0c4628dbd1415411480d7e9453f4a0096bab4edacfee77ab3b0ceed96fc4fd WatchSource:0}: Error finding container 3b0c4628dbd1415411480d7e9453f4a0096bab4edacfee77ab3b0ceed96fc4fd: Status 404 returned error can't find the container with id 3b0c4628dbd1415411480d7e9453f4a0096bab4edacfee77ab3b0ceed96fc4fd
Jul 09 22:16:58.478777 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:58.478741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp" event={"ID":"969cc86e-0ec0-42e1-affb-539139f620e0","Type":"ContainerStarted","Data":"11c202c14c24bbf144439ecd288f3ce5da87e839ddf07847f0049fddf3e22aa9"}
Jul 09 22:16:58.478777 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:58.478781    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp" event={"ID":"969cc86e-0ec0-42e1-affb-539139f620e0","Type":"ContainerStarted","Data":"3b0c4628dbd1415411480d7e9453f4a0096bab4edacfee77ab3b0ceed96fc4fd"}
Jul 09 22:16:58.479269 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:58.478887    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:16:58.494737 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:16:58.494684    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp" podStartSLOduration=1.494669057 podStartE2EDuration="1.494669057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:16:58.493783373 +0000 UTC m=+498.384813887" watchObservedRunningTime="2026-07-09 22:16:58.494669057 +0000 UTC m=+498.385699570"
Jul 09 22:17:02.384280 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.384230    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"]
Jul 09 22:17:02.387464 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.387448    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.389537 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.389519    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 22:17:02.389649 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.389521    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 22:17:02.390076 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.390052    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-q4vvd\""
Jul 09 22:17:02.397085 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.397061    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"]
Jul 09 22:17:02.433940 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.433911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55914045-8562-4de3-aa9e-e6a7590736df-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m9sj6\" (UID: \"55914045-8562-4de3-aa9e-e6a7590736df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.434068 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.433944    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqssr\" (UniqueName: \"kubernetes.io/projected/55914045-8562-4de3-aa9e-e6a7590736df-kube-api-access-vqssr\") pod \"openshift-lws-operator-fd8ccff4c-m9sj6\" (UID: \"55914045-8562-4de3-aa9e-e6a7590736df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.535336 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.535300    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55914045-8562-4de3-aa9e-e6a7590736df-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m9sj6\" (UID: \"55914045-8562-4de3-aa9e-e6a7590736df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.535504 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.535341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqssr\" (UniqueName: \"kubernetes.io/projected/55914045-8562-4de3-aa9e-e6a7590736df-kube-api-access-vqssr\") pod \"openshift-lws-operator-fd8ccff4c-m9sj6\" (UID: \"55914045-8562-4de3-aa9e-e6a7590736df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.535686 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.535666    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55914045-8562-4de3-aa9e-e6a7590736df-tmp\") pod \"openshift-lws-operator-fd8ccff4c-m9sj6\" (UID: \"55914045-8562-4de3-aa9e-e6a7590736df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.544872 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.544852    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqssr\" (UniqueName: \"kubernetes.io/projected/55914045-8562-4de3-aa9e-e6a7590736df-kube-api-access-vqssr\") pod \"openshift-lws-operator-fd8ccff4c-m9sj6\" (UID: \"55914045-8562-4de3-aa9e-e6a7590736df\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.697375 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.697287    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"
Jul 09 22:17:02.846803 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:02.846778    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6"]
Jul 09 22:17:02.849452 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:17:02.849422    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55914045_8562_4de3_aa9e_e6a7590736df.slice/crio-6a0eca5cf5fef54efc408906875cbc287632d7ad11379efb2638095d813f8e6b WatchSource:0}: Error finding container 6a0eca5cf5fef54efc408906875cbc287632d7ad11379efb2638095d813f8e6b: Status 404 returned error can't find the container with id 6a0eca5cf5fef54efc408906875cbc287632d7ad11379efb2638095d813f8e6b
Jul 09 22:17:03.502181 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:03.502143    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6" event={"ID":"55914045-8562-4de3-aa9e-e6a7590736df","Type":"ContainerStarted","Data":"6a0eca5cf5fef54efc408906875cbc287632d7ad11379efb2638095d813f8e6b"}
Jul 09 22:17:04.486317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:04.486280    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-ctrxp"
Jul 09 22:17:06.520099 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:06.520048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6" event={"ID":"55914045-8562-4de3-aa9e-e6a7590736df","Type":"ContainerStarted","Data":"f4b67e7c8046e8da45eb1d842867325cc805b5b60a25e2d9b9e3ed0ebd16fe0c"}
Jul 09 22:17:06.537363 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:06.537317    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-m9sj6" podStartSLOduration=1.7439865970000001 podStartE2EDuration="4.537301096s" totalImagesPullingTime="2.793314499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:06.535335136 +0000 UTC m=+506.426365650" watchObservedRunningTime="2026-07-09 22:17:06.537301096 +0000 UTC m=+506.428331731"
Jul 09 22:17:22.744624 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.744587    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"]
Jul 09 22:17:22.748304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.748286    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:22.754221 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.754195    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 22:17:22.754362 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.754218    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 22:17:22.754421 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.754403    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 22:17:22.754479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.754427    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 22:17:22.754479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.754461    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kj66l\""
Jul 09 22:17:22.797242 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.797160    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"]
Jul 09 22:17:22.916828 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.916798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcllh\" (UniqueName: \"kubernetes.io/projected/98aa773d-8adf-4867-8614-86ed24c814fb-kube-api-access-dcllh\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:22.917088 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.916868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98aa773d-8adf-4867-8614-86ed24c814fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:22.917088 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.916962    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/98aa773d-8adf-4867-8614-86ed24c814fb-manifests\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:22.917088 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.917002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98aa773d-8adf-4867-8614-86ed24c814fb-tmp\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:22.917088 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:22.917041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/98aa773d-8adf-4867-8614-86ed24c814fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.017760 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.017674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/98aa773d-8adf-4867-8614-86ed24c814fb-manifests\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.017760 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.017708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98aa773d-8adf-4867-8614-86ed24c814fb-tmp\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.017760 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.017727    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/98aa773d-8adf-4867-8614-86ed24c814fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.018038 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.017865    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcllh\" (UniqueName: \"kubernetes.io/projected/98aa773d-8adf-4867-8614-86ed24c814fb-kube-api-access-dcllh\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.018038 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.017999    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98aa773d-8adf-4867-8614-86ed24c814fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.018148 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.018107    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/98aa773d-8adf-4867-8614-86ed24c814fb-manifests\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.020071 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.020039    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98aa773d-8adf-4867-8614-86ed24c814fb-tmp\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.020294 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.020275    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/98aa773d-8adf-4867-8614-86ed24c814fb-webhook-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.020418 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.020398    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/98aa773d-8adf-4867-8614-86ed24c814fb-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.027794 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.027766    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcllh\" (UniqueName: \"kubernetes.io/projected/98aa773d-8adf-4867-8614-86ed24c814fb-kube-api-access-dcllh\") pod \"opendatahub-operator-controller-manager-5b79867d65-fpkzd\" (UID: \"98aa773d-8adf-4867-8614-86ed24c814fb\") " pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.058980 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.058954    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:23.268107 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.268059    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"]
Jul 09 22:17:23.270671 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:17:23.270644    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98aa773d_8adf_4867_8614_86ed24c814fb.slice/crio-26d5cc7519e96806462991a62208e03dc9fd2f184660b2c12db1f873da2ed7c8 WatchSource:0}: Error finding container 26d5cc7519e96806462991a62208e03dc9fd2f184660b2c12db1f873da2ed7c8: Status 404 returned error can't find the container with id 26d5cc7519e96806462991a62208e03dc9fd2f184660b2c12db1f873da2ed7c8
Jul 09 22:17:23.608054 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:23.607966    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd" event={"ID":"98aa773d-8adf-4867-8614-86ed24c814fb","Type":"ContainerStarted","Data":"26d5cc7519e96806462991a62208e03dc9fd2f184660b2c12db1f873da2ed7c8"}
Jul 09 22:17:26.623503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:26.623422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd" event={"ID":"98aa773d-8adf-4867-8614-86ed24c814fb","Type":"ContainerStarted","Data":"888beb6e3b3d29906b684207cae84cc759d6c79cc25ec372bd232580a7364cfd"}
Jul 09 22:17:27.629144 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:27.629106    2566 generic.go:356] "Generic (PLEG): container finished" podID="98aa773d-8adf-4867-8614-86ed24c814fb" containerID="888beb6e3b3d29906b684207cae84cc759d6c79cc25ec372bd232580a7364cfd" exitCode=0
Jul 09 22:17:27.629638 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:27.629193    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd" event={"ID":"98aa773d-8adf-4867-8614-86ed24c814fb","Type":"ContainerDied","Data":"888beb6e3b3d29906b684207cae84cc759d6c79cc25ec372bd232580a7364cfd"}
Jul 09 22:17:28.635419 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:28.635386    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd" event={"ID":"98aa773d-8adf-4867-8614-86ed24c814fb","Type":"ContainerStarted","Data":"0011641ae1383eeb4a4b4ff20bdc2a59c11bf039618926d3416beddd343946ae"}
Jul 09 22:17:28.635828 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:28.635506    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:28.668582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:28.668500    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd" podStartSLOduration=2.204292891 podStartE2EDuration="6.668480267s" totalImagesPullingTime="3.427984233s" totalInitContainerRuntime="1.036203143s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:28.668056095 +0000 UTC m=+528.559086622" watchObservedRunningTime="2026-07-09 22:17:28.668480267 +0000 UTC m=+528.559510782"
Jul 09 22:17:39.642195 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:39.642117    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5b79867d65-fpkzd"
Jul 09 22:17:42.356936 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.356899    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"]
Jul 09 22:17:42.362572 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.362547    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.368135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.368112    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n55m9\""
Jul 09 22:17:42.369774 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.369757    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 22:17:42.370329 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.370314    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 22:17:42.383311 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:17:42.383274    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-lws-operator\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" type="*v1.Secret"
Jul 09 22:17:42.388063 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.388039    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4b2t\" (UniqueName: \"kubernetes.io/projected/37d0979b-1ccb-45f9-9eb2-23e417511e74-kube-api-access-p4b2t\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.388178 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.388097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37d0979b-1ccb-45f9-9eb2-23e417511e74-manager-config\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.388178 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.388151    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.388178 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.388172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-metrics-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.420409 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.420380    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"]
Jul 09 22:17:42.488902 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.488863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4b2t\" (UniqueName: \"kubernetes.io/projected/37d0979b-1ccb-45f9-9eb2-23e417511e74-kube-api-access-p4b2t\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.489081 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.488924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37d0979b-1ccb-45f9-9eb2-23e417511e74-manager-config\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.489081 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.488983    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.489081 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.489016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-metrics-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.489679 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.489658    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37d0979b-1ccb-45f9-9eb2-23e417511e74-manager-config\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.491569 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.491546    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-metrics-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:42.525683 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:42.525659    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4b2t\" (UniqueName: \"kubernetes.io/projected/37d0979b-1ccb-45f9-9eb2-23e417511e74-kube-api-access-p4b2t\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:43.350358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.350319    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"]
Jul 09 22:17:43.352766 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.352750    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.381682 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.381644    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 22:17:43.383979 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.383942    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 22:17:43.383979 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.383959    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 22:17:43.384329 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.384270    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l52kc\""
Jul 09 22:17:43.384543 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.384521    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"]
Jul 09 22:17:43.396788 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.396751    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adfeb335-b804-4bc3-92e0-2ca883fa0534-tmp\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.396977 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.396832    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28vjj\" (UniqueName: \"kubernetes.io/projected/adfeb335-b804-4bc3-92e0-2ca883fa0534-kube-api-access-28vjj\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.396977 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.396919    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adfeb335-b804-4bc3-92e0-2ca883fa0534-tls-certs\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.436426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.436382    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 22:17:43.490069 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:17:43.490028    2566 secret.go:189] Couldn't get secret openshift-lws-operator/webhook-server-cert: failed to sync secret cache: timed out waiting for the condition
Jul 09 22:17:43.490289 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:17:43.490124    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-cert podName:37d0979b-1ccb-45f9-9eb2-23e417511e74 nodeName:}" failed. No retries permitted until 2026-07-09 22:17:43.990106225 +0000 UTC m=+543.881136717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-cert") pod "lws-controller-manager-55bc84657b-47kcx" (UID: "37d0979b-1ccb-45f9-9eb2-23e417511e74") : failed to sync secret cache: timed out waiting for the condition
Jul 09 22:17:43.497508 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.497475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adfeb335-b804-4bc3-92e0-2ca883fa0534-tls-certs\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.497594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.497544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adfeb335-b804-4bc3-92e0-2ca883fa0534-tmp\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.497594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.497575    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28vjj\" (UniqueName: \"kubernetes.io/projected/adfeb335-b804-4bc3-92e0-2ca883fa0534-kube-api-access-28vjj\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.500604 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.500570    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adfeb335-b804-4bc3-92e0-2ca883fa0534-tmp\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.500805 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.500786    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/adfeb335-b804-4bc3-92e0-2ca883fa0534-tls-certs\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.581986 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.581944    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28vjj\" (UniqueName: \"kubernetes.io/projected/adfeb335-b804-4bc3-92e0-2ca883fa0534-kube-api-access-28vjj\") pod \"kube-auth-proxy-7ff5d987cc-g2kn8\" (UID: \"adfeb335-b804-4bc3-92e0-2ca883fa0534\") " pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.663569 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.663469    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"
Jul 09 22:17:43.933684 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.933587    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 22:17:43.968760 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:43.968721    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8"]
Jul 09 22:17:43.975581 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:17:43.975553    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadfeb335_b804_4bc3_92e0_2ca883fa0534.slice/crio-c5572e5d85ae7fb1a79b43d760509fb469297e1699b1ac2803272179a4cc0f92 WatchSource:0}: Error finding container c5572e5d85ae7fb1a79b43d760509fb469297e1699b1ac2803272179a4cc0f92: Status 404 returned error can't find the container with id c5572e5d85ae7fb1a79b43d760509fb469297e1699b1ac2803272179a4cc0f92
Jul 09 22:17:44.001689 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:44.001651    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:44.004093 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:44.004070    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d0979b-1ccb-45f9-9eb2-23e417511e74-cert\") pod \"lws-controller-manager-55bc84657b-47kcx\" (UID: \"37d0979b-1ccb-45f9-9eb2-23e417511e74\") " pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:44.172005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:44.171972    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:44.334445 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:44.334407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"]
Jul 09 22:17:44.337362 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:17:44.337332    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d0979b_1ccb_45f9_9eb2_23e417511e74.slice/crio-7bd3608e1c670968432f26e2b111a7201a3b61df23d11861be7f603586fe2bd6 WatchSource:0}: Error finding container 7bd3608e1c670968432f26e2b111a7201a3b61df23d11861be7f603586fe2bd6: Status 404 returned error can't find the container with id 7bd3608e1c670968432f26e2b111a7201a3b61df23d11861be7f603586fe2bd6
Jul 09 22:17:44.714333 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:44.714292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx" event={"ID":"37d0979b-1ccb-45f9-9eb2-23e417511e74","Type":"ContainerStarted","Data":"7bd3608e1c670968432f26e2b111a7201a3b61df23d11861be7f603586fe2bd6"}
Jul 09 22:17:44.715763 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:44.715733    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8" event={"ID":"adfeb335-b804-4bc3-92e0-2ca883fa0534","Type":"ContainerStarted","Data":"c5572e5d85ae7fb1a79b43d760509fb469297e1699b1ac2803272179a4cc0f92"}
Jul 09 22:17:47.740974 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:47.740932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx" event={"ID":"37d0979b-1ccb-45f9-9eb2-23e417511e74","Type":"ContainerStarted","Data":"8d25e704f493528cf355f168f5798ca77a0a58e2ef95614159abc877c5e30817"}
Jul 09 22:17:47.741481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:47.741109    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:17:47.742771 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:47.742741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8" event={"ID":"adfeb335-b804-4bc3-92e0-2ca883fa0534","Type":"ContainerStarted","Data":"acc9399ce906f9c4ffb73579df0d5ceeb8261cf5f9354c1b1b914719796606eb"}
Jul 09 22:17:47.787945 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:47.787892    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx" podStartSLOduration=2.95907746 podStartE2EDuration="5.787873202s" totalImagesPullingTime="2.828795742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:47.785851497 +0000 UTC m=+547.676882010" watchObservedRunningTime="2026-07-09 22:17:47.787873202 +0000 UTC m=+547.678903715"
Jul 09 22:17:47.840409 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:47.840343    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7ff5d987cc-g2kn8" podStartSLOduration=1.643252052 podStartE2EDuration="4.840324759s" totalImagesPullingTime="3.197072707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:17:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:17:47.839162671 +0000 UTC m=+547.730193187" watchObservedRunningTime="2026-07-09 22:17:47.840324759 +0000 UTC m=+547.731355276"
Jul 09 22:17:58.750326 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:17:58.750290    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55bc84657b-47kcx"
Jul 09 22:19:32.006425 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.006385    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"]
Jul 09 22:19:32.010057 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.010036    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.012758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.012735    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 09 22:19:32.012877 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.012835    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 09 22:19:32.012979 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.012963    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qxmpn\""
Jul 09 22:19:32.013459 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.013445    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 22:19:32.013527 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.013492    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 22:19:32.021034 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.021010    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"]
Jul 09 22:19:32.096877 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.096837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/029c032c-df11-44ac-815e-59be560a8f31-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.097080 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.096970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/029c032c-df11-44ac-815e-59be560a8f31-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.097080 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.097001    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtv9m\" (UniqueName: \"kubernetes.io/projected/029c032c-df11-44ac-815e-59be560a8f31-kube-api-access-dtv9m\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.197554 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.197509    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/029c032c-df11-44ac-815e-59be560a8f31-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.197765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.197589    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/029c032c-df11-44ac-815e-59be560a8f31-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.197765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.197606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtv9m\" (UniqueName: \"kubernetes.io/projected/029c032c-df11-44ac-815e-59be560a8f31-kube-api-access-dtv9m\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.197765 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:19:32.197667    2566 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 09 22:19:32.197927 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:19:32.197771    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/029c032c-df11-44ac-815e-59be560a8f31-plugin-serving-cert podName:029c032c-df11-44ac-815e-59be560a8f31 nodeName:}" failed. No retries permitted until 2026-07-09 22:19:32.697751661 +0000 UTC m=+652.588782159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/029c032c-df11-44ac-815e-59be560a8f31-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-r2dxt" (UID: "029c032c-df11-44ac-815e-59be560a8f31") : secret "plugin-serving-cert" not found
Jul 09 22:19:32.198268 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.198220    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/029c032c-df11-44ac-815e-59be560a8f31-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.206808 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.206784    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtv9m\" (UniqueName: \"kubernetes.io/projected/029c032c-df11-44ac-815e-59be560a8f31-kube-api-access-dtv9m\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.701501 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.701462    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/029c032c-df11-44ac-815e-59be560a8f31-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.704011 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.703985    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/029c032c-df11-44ac-815e-59be560a8f31-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-r2dxt\" (UID: \"029c032c-df11-44ac-815e-59be560a8f31\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:32.920375 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:32.920338    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"
Jul 09 22:19:33.060275 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:33.060224    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt"]
Jul 09 22:19:33.062464 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:19:33.062438    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod029c032c_df11_44ac_815e_59be560a8f31.slice/crio-0f1894c510ee60442fe264c36f41a718ab0dcf58808507cb837fe36e9ed22601 WatchSource:0}: Error finding container 0f1894c510ee60442fe264c36f41a718ab0dcf58808507cb837fe36e9ed22601: Status 404 returned error can't find the container with id 0f1894c510ee60442fe264c36f41a718ab0dcf58808507cb837fe36e9ed22601
Jul 09 22:19:33.063843 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:33.063827    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 22:19:33.243112 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:33.243021    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt" event={"ID":"029c032c-df11-44ac-815e-59be560a8f31","Type":"ContainerStarted","Data":"0f1894c510ee60442fe264c36f41a718ab0dcf58808507cb837fe36e9ed22601"}
Jul 09 22:19:56.388480 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:19:56.388441    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt" event={"ID":"029c032c-df11-44ac-815e-59be560a8f31","Type":"ContainerStarted","Data":"7509ec2766b9680187185b947f93280a4f33cffb4de340714e2eefea4e0030e6"}
Jul 09 22:20:16.749080 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.749026    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-r2dxt" podStartSLOduration=22.500098566 podStartE2EDuration="45.74900951s" totalImagesPullingTime="23.248910944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:19:56.421406608 +0000 UTC m=+676.312437123" watchObservedRunningTime="2026-07-09 22:20:16.74900951 +0000 UTC m=+696.640040023"
Jul 09 22:20:16.750075 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.750057    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-585cb75754-z5nvn"]
Jul 09 22:20:16.814840 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.814807    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-585cb75754-z5nvn"]
Jul 09 22:20:16.815064 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.814919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.919813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.919779    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-oauth-serving-cert\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.919813 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.919826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-trusted-ca-bundle\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.920095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.919850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-oauth-config\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.920095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.919966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-config\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.920095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.920030    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjsrz\" (UniqueName: \"kubernetes.io/projected/6cc3cb92-0c1b-4d10-82be-ec18d318396d-kube-api-access-rjsrz\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.920095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.920070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-serving-cert\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:16.920285 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:16.920112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-service-ca\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021508 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021410    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-oauth-config\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021508 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021467    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-config\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021508 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjsrz\" (UniqueName: \"kubernetes.io/projected/6cc3cb92-0c1b-4d10-82be-ec18d318396d-kube-api-access-rjsrz\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-serving-cert\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-service-ca\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-oauth-serving-cert\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.021789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.021663    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-trusted-ca-bundle\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.022490 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.022462    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-service-ca\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.022608 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.022486    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-config\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.022608 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.022462    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-oauth-serving-cert\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.022608 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.022549    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6cc3cb92-0c1b-4d10-82be-ec18d318396d-trusted-ca-bundle\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.023974 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.023954    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-oauth-config\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.024114 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.024096    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cc3cb92-0c1b-4d10-82be-ec18d318396d-console-serving-cert\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.031241 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.031214    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjsrz\" (UniqueName: \"kubernetes.io/projected/6cc3cb92-0c1b-4d10-82be-ec18d318396d-kube-api-access-rjsrz\") pod \"console-585cb75754-z5nvn\" (UID: \"6cc3cb92-0c1b-4d10-82be-ec18d318396d\") " pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.124955 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.124918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:17.285739 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.285714    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-585cb75754-z5nvn"]
Jul 09 22:20:17.288334 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:20:17.288312    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc3cb92_0c1b_4d10_82be_ec18d318396d.slice/crio-b4ce7672028f18253c05b02c5143bb1661ddfc7ac0b0104e47ed6240bf56610d WatchSource:0}: Error finding container b4ce7672028f18253c05b02c5143bb1661ddfc7ac0b0104e47ed6240bf56610d: Status 404 returned error can't find the container with id b4ce7672028f18253c05b02c5143bb1661ddfc7ac0b0104e47ed6240bf56610d
Jul 09 22:20:17.493845 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.493807    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-585cb75754-z5nvn" event={"ID":"6cc3cb92-0c1b-4d10-82be-ec18d318396d","Type":"ContainerStarted","Data":"105f9ed909477a00b653153c90c8ef38548944a745c80960d691b287045da262"}
Jul 09 22:20:17.494030 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.493898    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-585cb75754-z5nvn" event={"ID":"6cc3cb92-0c1b-4d10-82be-ec18d318396d","Type":"ContainerStarted","Data":"b4ce7672028f18253c05b02c5143bb1661ddfc7ac0b0104e47ed6240bf56610d"}
Jul 09 22:20:17.513075 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:17.513007    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-585cb75754-z5nvn" podStartSLOduration=1.512988364 podStartE2EDuration="1.512988364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:17.512409779 +0000 UTC m=+697.403440292" watchObservedRunningTime="2026-07-09 22:20:17.512988364 +0000 UTC m=+697.404018879"
Jul 09 22:20:20.410935 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.410896    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:20:20.473736 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.473695    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:20:20.473736 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.473737    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:20:20.473953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.473827    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.476200 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.476180    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 22:20:20.656345 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.656298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/000ca2f2-1f10-4662-a485-329ff5a23539-config-file\") pod \"limitador-limitador-54d6d988d5-lnjkn\" (UID: \"000ca2f2-1f10-4662-a485-329ff5a23539\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.656522 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.656431    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnqkp\" (UniqueName: \"kubernetes.io/projected/000ca2f2-1f10-4662-a485-329ff5a23539-kube-api-access-mnqkp\") pod \"limitador-limitador-54d6d988d5-lnjkn\" (UID: \"000ca2f2-1f10-4662-a485-329ff5a23539\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.757350 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.757303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnqkp\" (UniqueName: \"kubernetes.io/projected/000ca2f2-1f10-4662-a485-329ff5a23539-kube-api-access-mnqkp\") pod \"limitador-limitador-54d6d988d5-lnjkn\" (UID: \"000ca2f2-1f10-4662-a485-329ff5a23539\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.757561 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.757397    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/000ca2f2-1f10-4662-a485-329ff5a23539-config-file\") pod \"limitador-limitador-54d6d988d5-lnjkn\" (UID: \"000ca2f2-1f10-4662-a485-329ff5a23539\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.757996 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.757979    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/000ca2f2-1f10-4662-a485-329ff5a23539-config-file\") pod \"limitador-limitador-54d6d988d5-lnjkn\" (UID: \"000ca2f2-1f10-4662-a485-329ff5a23539\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.767061 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.767027    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnqkp\" (UniqueName: \"kubernetes.io/projected/000ca2f2-1f10-4662-a485-329ff5a23539-kube-api-access-mnqkp\") pod \"limitador-limitador-54d6d988d5-lnjkn\" (UID: \"000ca2f2-1f10-4662-a485-329ff5a23539\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.784743 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.784710    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:20.931906 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:20.931877    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:20:20.934197 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:20:20.934163    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod000ca2f2_1f10_4662_a485_329ff5a23539.slice/crio-38d1c6304a502b74d5aa1d434dd4fd5b04563f9550fab11582554135e9ac7212 WatchSource:0}: Error finding container 38d1c6304a502b74d5aa1d434dd4fd5b04563f9550fab11582554135e9ac7212: Status 404 returned error can't find the container with id 38d1c6304a502b74d5aa1d434dd4fd5b04563f9550fab11582554135e9ac7212
Jul 09 22:20:21.024414 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.024322    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tkr8t"]
Jul 09 22:20:21.029154 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.029134    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:21.031792 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.031762    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7sbg9\""
Jul 09 22:20:21.033535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.033506    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tkr8t"]
Jul 09 22:20:21.160138 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.160098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrrmj\" (UniqueName: \"kubernetes.io/projected/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db-kube-api-access-zrrmj\") pod \"authorino-75d556bd77-tkr8t\" (UID: \"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db\") " pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:21.261007 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.260970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrrmj\" (UniqueName: \"kubernetes.io/projected/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db-kube-api-access-zrrmj\") pod \"authorino-75d556bd77-tkr8t\" (UID: \"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db\") " pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:21.269514 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.269488    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrrmj\" (UniqueName: \"kubernetes.io/projected/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db-kube-api-access-zrrmj\") pod \"authorino-75d556bd77-tkr8t\" (UID: \"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db\") " pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:21.339349 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.339242    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:21.487576 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.487524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tkr8t"]
Jul 09 22:20:21.491008 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:20:21.490975    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ebd7cc3_8716_4c4c_ab3a_6290a4d055db.slice/crio-d3f13cbd27b2f0be67a83bd5a2e23d3148a94e9163ca2a63d46d45152107e264 WatchSource:0}: Error finding container d3f13cbd27b2f0be67a83bd5a2e23d3148a94e9163ca2a63d46d45152107e264: Status 404 returned error can't find the container with id d3f13cbd27b2f0be67a83bd5a2e23d3148a94e9163ca2a63d46d45152107e264
Jul 09 22:20:21.519553 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.519474    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tkr8t" event={"ID":"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db","Type":"ContainerStarted","Data":"d3f13cbd27b2f0be67a83bd5a2e23d3148a94e9163ca2a63d46d45152107e264"}
Jul 09 22:20:21.521909 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:21.521799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn" event={"ID":"000ca2f2-1f10-4662-a485-329ff5a23539","Type":"ContainerStarted","Data":"38d1c6304a502b74d5aa1d434dd4fd5b04563f9550fab11582554135e9ac7212"}
Jul 09 22:20:25.551018 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:25.550931    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn" event={"ID":"000ca2f2-1f10-4662-a485-329ff5a23539","Type":"ContainerStarted","Data":"b02e1fcb2b968c1953bc29633e830f9c4d1bc502758bc4476cbdeb888156ebd8"}
Jul 09 22:20:25.551624 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:25.551598    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:25.552860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:25.552835    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tkr8t" event={"ID":"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db","Type":"ContainerStarted","Data":"1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e"}
Jul 09 22:20:25.572165 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:25.572114    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn" podStartSLOduration=1.24818739 podStartE2EDuration="5.572100806s" totalImagesPullingTime="4.323913416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:25.569681917 +0000 UTC m=+705.460712432" watchObservedRunningTime="2026-07-09 22:20:25.572100806 +0000 UTC m=+705.463131320"
Jul 09 22:20:25.585699 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:25.585654    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tkr8t" podStartSLOduration=0.758508449 podStartE2EDuration="4.585640632s" totalImagesPullingTime="3.827132183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:25.583672641 +0000 UTC m=+705.474703155" watchObservedRunningTime="2026-07-09 22:20:25.585640632 +0000 UTC m=+705.476671146"
Jul 09 22:20:27.125862 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:27.125828    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:27.126284 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:27.125887    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:27.130612 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:27.130590    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:27.566554 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:27.566517    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-585cb75754-z5nvn"
Jul 09 22:20:27.610535 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:27.610506    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659897f855-tc4kk"]
Jul 09 22:20:36.559268 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:36.559171    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"
Jul 09 22:20:52.633004 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:52.632952    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-659897f855-tc4kk" podUID="3e042d50-5e80-4a5e-96dc-6f5b842f5a07" containerName="console" containerID="cri-o://329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c" gracePeriod=15
Jul 09 22:20:52.891991 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:52.891925    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:20:53.067560 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067527    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-trusted-ca-bundle\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.067765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067576    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-oauth-serving-cert\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.067765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067608    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-service-ca\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.067765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067644    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-serving-cert\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.067765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067712    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-oauth-config\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.067765 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067735    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-config\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.068005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.067823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qllmr\" (UniqueName: \"kubernetes.io/projected/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-kube-api-access-qllmr\") pod \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\" (UID: \"3e042d50-5e80-4a5e-96dc-6f5b842f5a07\") "
Jul 09 22:20:53.068054 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.068026    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:20:53.068103 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.068075    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-service-ca" (OuterVolumeSpecName: "service-ca") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:20:53.068154 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.068091    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:20:53.068154 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.068129    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-trusted-ca-bundle\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.068466 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.068444    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-config" (OuterVolumeSpecName: "console-config") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 22:20:53.070001 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.069970    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:20:53.070095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.070007    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-kube-api-access-qllmr" (OuterVolumeSpecName: "kube-api-access-qllmr") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "kube-api-access-qllmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:20:53.070095 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.070049    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3e042d50-5e80-4a5e-96dc-6f5b842f5a07" (UID: "3e042d50-5e80-4a5e-96dc-6f5b842f5a07"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:20:53.169600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.169513    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-oauth-config\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.169600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.169545    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-config\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.169600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.169555    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qllmr\" (UniqueName: \"kubernetes.io/projected/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-kube-api-access-qllmr\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.169600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.169565    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-oauth-serving-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.169600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.169574    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-service-ca\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.169600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.169583    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3e042d50-5e80-4a5e-96dc-6f5b842f5a07-console-serving-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:53.701031 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.700999    2566 generic.go:356] "Generic (PLEG): container finished" podID="3e042d50-5e80-4a5e-96dc-6f5b842f5a07" containerID="329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c" exitCode=0
Jul 09 22:20:53.701492 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.701049    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659897f855-tc4kk"
Jul 09 22:20:53.701492 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.701080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659897f855-tc4kk" event={"ID":"3e042d50-5e80-4a5e-96dc-6f5b842f5a07","Type":"ContainerDied","Data":"329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c"}
Jul 09 22:20:53.701492 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.701133    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659897f855-tc4kk" event={"ID":"3e042d50-5e80-4a5e-96dc-6f5b842f5a07","Type":"ContainerDied","Data":"9d95fdc49b09fda2eece24032d9ef3210f74b059beb8cd12e4bca1b4b49e95ba"}
Jul 09 22:20:53.701492 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.701147    2566 scope.go:117] "RemoveContainer" containerID="329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c"
Jul 09 22:20:53.709874 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.709855    2566 scope.go:117] "RemoveContainer" containerID="329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c"
Jul 09 22:20:53.710127 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:20:53.710106    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c\": container with ID starting with 329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c not found: ID does not exist" containerID="329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c"
Jul 09 22:20:53.710168 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.710141    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c"} err="failed to get container status \"329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c\": rpc error: code = NotFound desc = could not find container \"329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c\": container with ID starting with 329a4bd785c04f2eda12a1d9374d0837c4201e52c46e0939ce2924010d77079c not found: ID does not exist"
Jul 09 22:20:53.718607 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.718583    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659897f855-tc4kk"]
Jul 09 22:20:53.722040 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:53.722018    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-659897f855-tc4kk"]
Jul 09 22:20:54.687009 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.686972    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g47hz"]
Jul 09 22:20:54.687384 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.687370    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e042d50-5e80-4a5e-96dc-6f5b842f5a07" containerName="console"
Jul 09 22:20:54.687432 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.687386    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e042d50-5e80-4a5e-96dc-6f5b842f5a07" containerName="console"
Jul 09 22:20:54.687466 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.687455    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e042d50-5e80-4a5e-96dc-6f5b842f5a07" containerName="console"
Jul 09 22:20:54.739260 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.739209    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g47hz"]
Jul 09 22:20:54.739640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.739354    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:54.875713 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.875667    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g47hz"]
Jul 09 22:20:54.876025 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:20:54.875999    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-z7v25], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-g47hz" podUID="ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6"
Jul 09 22:20:54.886079 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.886045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7v25\" (UniqueName: \"kubernetes.io/projected/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6-kube-api-access-z7v25\") pod \"authorino-5d4b67c446-g47hz\" (UID: \"ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6\") " pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:54.903642 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.903605    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-bvl9v"]
Jul 09 22:20:54.922505 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.922471    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-bvl9v"]
Jul 09 22:20:54.922690 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.922587    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:54.924896 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.924872    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 22:20:54.965490 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.965455    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-bvl9v"]
Jul 09 22:20:54.965781 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:20:54.965759    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-gf9cr tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-gf9cr tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-bvl9v" podUID="8e9675a4-bb19-459a-9259-b1b1059d3496"
Jul 09 22:20:54.986639 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.986603    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7v25\" (UniqueName: \"kubernetes.io/projected/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6-kube-api-access-z7v25\") pod \"authorino-5d4b67c446-g47hz\" (UID: \"ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6\") " pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:54.994408 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.994378    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b5f889966-58v6v"]
Jul 09 22:20:54.997124 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:54.997100    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7v25\" (UniqueName: \"kubernetes.io/projected/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6-kube-api-access-z7v25\") pod \"authorino-5d4b67c446-g47hz\" (UID: \"ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6\") " pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:55.020434 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.020361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b5f889966-58v6v"]
Jul 09 22:20:55.020602 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.020454    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.087694 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.087651    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e9675a4-bb19-459a-9259-b1b1059d3496-tls-cert\") pod \"authorino-76877994-bvl9v\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") " pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.087893 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.087727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf9cr\" (UniqueName: \"kubernetes.io/projected/8e9675a4-bb19-459a-9259-b1b1059d3496-kube-api-access-gf9cr\") pod \"authorino-76877994-bvl9v\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") " pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.188321 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.188274    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pls9m\" (UniqueName: \"kubernetes.io/projected/783a6498-488a-4698-a324-9de03bf9929e-kube-api-access-pls9m\") pod \"authorino-b5f889966-58v6v\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") " pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.188321 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.188331    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e9675a4-bb19-459a-9259-b1b1059d3496-tls-cert\") pod \"authorino-76877994-bvl9v\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") " pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.188622 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.188383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf9cr\" (UniqueName: \"kubernetes.io/projected/8e9675a4-bb19-459a-9259-b1b1059d3496-kube-api-access-gf9cr\") pod \"authorino-76877994-bvl9v\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") " pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.188622 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.188405    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783a6498-488a-4698-a324-9de03bf9929e-tls-cert\") pod \"authorino-b5f889966-58v6v\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") " pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.190936 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.190912    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e9675a4-bb19-459a-9259-b1b1059d3496-tls-cert\") pod \"authorino-76877994-bvl9v\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") " pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.196852 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.196817    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf9cr\" (UniqueName: \"kubernetes.io/projected/8e9675a4-bb19-459a-9259-b1b1059d3496-kube-api-access-gf9cr\") pod \"authorino-76877994-bvl9v\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") " pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.289597 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.289506    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783a6498-488a-4698-a324-9de03bf9929e-tls-cert\") pod \"authorino-b5f889966-58v6v\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") " pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.289597 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.289591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pls9m\" (UniqueName: \"kubernetes.io/projected/783a6498-488a-4698-a324-9de03bf9929e-kube-api-access-pls9m\") pod \"authorino-b5f889966-58v6v\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") " pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.292135 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.292097    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783a6498-488a-4698-a324-9de03bf9929e-tls-cert\") pod \"authorino-b5f889966-58v6v\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") " pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.297739 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.297703    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pls9m\" (UniqueName: \"kubernetes.io/projected/783a6498-488a-4698-a324-9de03bf9929e-kube-api-access-pls9m\") pod \"authorino-b5f889966-58v6v\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") " pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.330720 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.330674    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:20:55.507351 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.507324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b5f889966-58v6v"]
Jul 09 22:20:55.509978 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:20:55.509950    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod783a6498_488a_4698_a324_9de03bf9929e.slice/crio-8497736ea3d1d0d60eacf3aa2165ae590415f6bdf7160c9f342d71c02874bd85 WatchSource:0}: Error finding container 8497736ea3d1d0d60eacf3aa2165ae590415f6bdf7160c9f342d71c02874bd85: Status 404 returned error can't find the container with id 8497736ea3d1d0d60eacf3aa2165ae590415f6bdf7160c9f342d71c02874bd85
Jul 09 22:20:55.680876 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.680761    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e042d50-5e80-4a5e-96dc-6f5b842f5a07" path="/var/lib/kubelet/pods/3e042d50-5e80-4a5e-96dc-6f5b842f5a07/volumes"
Jul 09 22:20:55.716571 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.716530    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f889966-58v6v" event={"ID":"783a6498-488a-4698-a324-9de03bf9929e","Type":"ContainerStarted","Data":"8497736ea3d1d0d60eacf3aa2165ae590415f6bdf7160c9f342d71c02874bd85"}
Jul 09 22:20:55.716571 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.716564    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:55.716571 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.716569    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.752733 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.752697    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:55.766805 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.766777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:55.895491 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.895458    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7v25\" (UniqueName: \"kubernetes.io/projected/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6-kube-api-access-z7v25\") pod \"ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6\" (UID: \"ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6\") "
Jul 09 22:20:55.895663 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.895525    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e9675a4-bb19-459a-9259-b1b1059d3496-tls-cert\") pod \"8e9675a4-bb19-459a-9259-b1b1059d3496\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") "
Jul 09 22:20:55.895663 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.895575    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf9cr\" (UniqueName: \"kubernetes.io/projected/8e9675a4-bb19-459a-9259-b1b1059d3496-kube-api-access-gf9cr\") pod \"8e9675a4-bb19-459a-9259-b1b1059d3496\" (UID: \"8e9675a4-bb19-459a-9259-b1b1059d3496\") "
Jul 09 22:20:55.897715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.897682    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6-kube-api-access-z7v25" (OuterVolumeSpecName: "kube-api-access-z7v25") pod "ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6" (UID: "ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6"). InnerVolumeSpecName "kube-api-access-z7v25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:20:55.897715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.897686    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e9675a4-bb19-459a-9259-b1b1059d3496-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8e9675a4-bb19-459a-9259-b1b1059d3496" (UID: "8e9675a4-bb19-459a-9259-b1b1059d3496"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:20:55.897864 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.897687    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e9675a4-bb19-459a-9259-b1b1059d3496-kube-api-access-gf9cr" (OuterVolumeSpecName: "kube-api-access-gf9cr") pod "8e9675a4-bb19-459a-9259-b1b1059d3496" (UID: "8e9675a4-bb19-459a-9259-b1b1059d3496"). InnerVolumeSpecName "kube-api-access-gf9cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:20:55.997181 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.997144    2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e9675a4-bb19-459a-9259-b1b1059d3496-tls-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:55.997181 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.997178    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf9cr\" (UniqueName: \"kubernetes.io/projected/8e9675a4-bb19-459a-9259-b1b1059d3496-kube-api-access-gf9cr\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:55.997446 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:55.997193    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7v25\" (UniqueName: \"kubernetes.io/projected/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6-kube-api-access-z7v25\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:56.724004 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.723964    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f889966-58v6v" event={"ID":"783a6498-488a-4698-a324-9de03bf9929e","Type":"ContainerStarted","Data":"317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179"}
Jul 09 22:20:56.724197 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.724063    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-bvl9v"
Jul 09 22:20:56.724197 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.724143    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g47hz"
Jul 09 22:20:56.740767 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.740724    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b5f889966-58v6v" podStartSLOduration=2.213355558 podStartE2EDuration="2.740709265s" totalImagesPullingTime="527.353707ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:20:56.73972398 +0000 UTC m=+736.630754494" watchObservedRunningTime="2026-07-09 22:20:56.740709265 +0000 UTC m=+736.631739779"
Jul 09 22:20:56.762302 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.762271    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g47hz"]
Jul 09 22:20:56.764122 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.764096    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g47hz"]
Jul 09 22:20:56.768465 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.768442    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tkr8t"]
Jul 09 22:20:56.768645 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.768623    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tkr8t" podUID="6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" containerName="authorino" containerID="cri-o://1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e" gracePeriod=30
Jul 09 22:20:56.788637 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.788604    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-bvl9v"]
Jul 09 22:20:56.794697 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:56.794666    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-bvl9v"]
Jul 09 22:20:57.020156 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.020132    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:57.109022 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.108989    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrrmj\" (UniqueName: \"kubernetes.io/projected/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db-kube-api-access-zrrmj\") pod \"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db\" (UID: \"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db\") "
Jul 09 22:20:57.111287 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.111264    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db-kube-api-access-zrrmj" (OuterVolumeSpecName: "kube-api-access-zrrmj") pod "6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" (UID: "6ebd7cc3-8716-4c4c-ab3a-6290a4d055db"). InnerVolumeSpecName "kube-api-access-zrrmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:20:57.210503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.210473    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrrmj\" (UniqueName: \"kubernetes.io/projected/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db-kube-api-access-zrrmj\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:20:57.679931 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.679896    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e9675a4-bb19-459a-9259-b1b1059d3496" path="/var/lib/kubelet/pods/8e9675a4-bb19-459a-9259-b1b1059d3496/volumes"
Jul 09 22:20:57.680166 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.680153    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6" path="/var/lib/kubelet/pods/ccf4f8cf-cfb2-4c09-9ba0-19561c6a24f6/volumes"
Jul 09 22:20:57.731336 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.731304    2566 generic.go:356] "Generic (PLEG): container finished" podID="6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" containerID="1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e" exitCode=0
Jul 09 22:20:57.731525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.731353    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tkr8t"
Jul 09 22:20:57.731525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.731384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tkr8t" event={"ID":"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db","Type":"ContainerDied","Data":"1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e"}
Jul 09 22:20:57.731525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.731442    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tkr8t" event={"ID":"6ebd7cc3-8716-4c4c-ab3a-6290a4d055db","Type":"ContainerDied","Data":"d3f13cbd27b2f0be67a83bd5a2e23d3148a94e9163ca2a63d46d45152107e264"}
Jul 09 22:20:57.731525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.731469    2566 scope.go:117] "RemoveContainer" containerID="1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e"
Jul 09 22:20:57.742302 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.742277    2566 scope.go:117] "RemoveContainer" containerID="1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e"
Jul 09 22:20:57.742627 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:20:57.742601    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e\": container with ID starting with 1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e not found: ID does not exist" containerID="1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e"
Jul 09 22:20:57.742685 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.742629    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e"} err="failed to get container status \"1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e\": rpc error: code = NotFound desc = could not find container \"1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e\": container with ID starting with 1efed8cd9f4cfffe5e364d778457325668e4183d0d1d868a52c4fe6fda25154e not found: ID does not exist"
Jul 09 22:20:57.746882 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.746852    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tkr8t"]
Jul 09 22:20:57.750666 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:57.750642    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tkr8t"]
Jul 09 22:20:59.683316 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:20:59.680673    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" path="/var/lib/kubelet/pods/6ebd7cc3-8716-4c4c-ab3a-6290a4d055db/volumes"
Jul 09 22:21:37.331527 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:21:37.331487    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:21:53.089627 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:21:53.089588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:21:58.321108 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:21:58.321066    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:22:07.389238 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:22:07.389192    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:22:20.887436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:22:20.887398    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:22:32.684740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:22:32.684700    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:22:52.285406 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:22:52.285365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:23:51.296457 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.296424    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c976bc6bc-v8m6q"]
Jul 09 22:23:51.296942 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.296802    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" containerName="authorino"
Jul 09 22:23:51.296942 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.296813    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" containerName="authorino"
Jul 09 22:23:51.296942 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.296874    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ebd7cc3-8716-4c4c-ab3a-6290a4d055db" containerName="authorino"
Jul 09 22:23:51.299723 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.299706    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.308197 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.308168    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c976bc6bc-v8m6q"]
Jul 09 22:23:51.488621 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.488584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/10105218-f51a-4c6a-a6e9-2f1e000ecda4-tls-cert\") pod \"authorino-7c976bc6bc-v8m6q\" (UID: \"10105218-f51a-4c6a-a6e9-2f1e000ecda4\") " pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.488791 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.488646    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk75t\" (UniqueName: \"kubernetes.io/projected/10105218-f51a-4c6a-a6e9-2f1e000ecda4-kube-api-access-zk75t\") pod \"authorino-7c976bc6bc-v8m6q\" (UID: \"10105218-f51a-4c6a-a6e9-2f1e000ecda4\") " pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.589758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.589667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/10105218-f51a-4c6a-a6e9-2f1e000ecda4-tls-cert\") pod \"authorino-7c976bc6bc-v8m6q\" (UID: \"10105218-f51a-4c6a-a6e9-2f1e000ecda4\") " pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.589758 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.589744    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk75t\" (UniqueName: \"kubernetes.io/projected/10105218-f51a-4c6a-a6e9-2f1e000ecda4-kube-api-access-zk75t\") pod \"authorino-7c976bc6bc-v8m6q\" (UID: \"10105218-f51a-4c6a-a6e9-2f1e000ecda4\") " pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.592165 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.592144    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/10105218-f51a-4c6a-a6e9-2f1e000ecda4-tls-cert\") pod \"authorino-7c976bc6bc-v8m6q\" (UID: \"10105218-f51a-4c6a-a6e9-2f1e000ecda4\") " pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.597942 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.597916    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk75t\" (UniqueName: \"kubernetes.io/projected/10105218-f51a-4c6a-a6e9-2f1e000ecda4-kube-api-access-zk75t\") pod \"authorino-7c976bc6bc-v8m6q\" (UID: \"10105218-f51a-4c6a-a6e9-2f1e000ecda4\") " pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.609794 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.609768    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c976bc6bc-v8m6q"
Jul 09 22:23:51.745973 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:51.745947    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c976bc6bc-v8m6q"]
Jul 09 22:23:51.748614 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:23:51.748585    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10105218_f51a_4c6a_a6e9_2f1e000ecda4.slice/crio-78fca9809a589d77afe8e2846d5171b5b6db788ec98143d5aba8d3be11130ac4 WatchSource:0}: Error finding container 78fca9809a589d77afe8e2846d5171b5b6db788ec98143d5aba8d3be11130ac4: Status 404 returned error can't find the container with id 78fca9809a589d77afe8e2846d5171b5b6db788ec98143d5aba8d3be11130ac4
Jul 09 22:23:52.573170 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:52.573128    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c976bc6bc-v8m6q" event={"ID":"10105218-f51a-4c6a-a6e9-2f1e000ecda4","Type":"ContainerStarted","Data":"4a275f404d79ead429491a15e68e4a8f30fa64db328cbd8f909a0ec57c54b974"}
Jul 09 22:23:52.573684 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:52.573178    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c976bc6bc-v8m6q" event={"ID":"10105218-f51a-4c6a-a6e9-2f1e000ecda4","Type":"ContainerStarted","Data":"78fca9809a589d77afe8e2846d5171b5b6db788ec98143d5aba8d3be11130ac4"}
Jul 09 22:23:52.590017 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:52.589957    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7c976bc6bc-v8m6q" podStartSLOduration=1.164183598 podStartE2EDuration="1.589936624s" totalImagesPullingTime="425.753026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:23:52.588956282 +0000 UTC m=+912.479986797" watchObservedRunningTime="2026-07-09 22:23:52.589936624 +0000 UTC m=+912.480967140"
Jul 09 22:23:52.619660 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:52.619618    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5f889966-58v6v"]
Jul 09 22:23:52.619836 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:52.619813    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-b5f889966-58v6v" podUID="783a6498-488a-4698-a324-9de03bf9929e" containerName="authorino" containerID="cri-o://317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179" gracePeriod=30
Jul 09 22:23:52.899333 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:52.899300    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:23:53.003238 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.003203    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783a6498-488a-4698-a324-9de03bf9929e-tls-cert\") pod \"783a6498-488a-4698-a324-9de03bf9929e\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") "
Jul 09 22:23:53.003413 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.003382    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pls9m\" (UniqueName: \"kubernetes.io/projected/783a6498-488a-4698-a324-9de03bf9929e-kube-api-access-pls9m\") pod \"783a6498-488a-4698-a324-9de03bf9929e\" (UID: \"783a6498-488a-4698-a324-9de03bf9929e\") "
Jul 09 22:23:53.005430 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.005394    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783a6498-488a-4698-a324-9de03bf9929e-kube-api-access-pls9m" (OuterVolumeSpecName: "kube-api-access-pls9m") pod "783a6498-488a-4698-a324-9de03bf9929e" (UID: "783a6498-488a-4698-a324-9de03bf9929e"). InnerVolumeSpecName "kube-api-access-pls9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:23:53.013907 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.013881    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/783a6498-488a-4698-a324-9de03bf9929e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "783a6498-488a-4698-a324-9de03bf9929e" (UID: "783a6498-488a-4698-a324-9de03bf9929e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:23:53.104618 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.104579    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pls9m\" (UniqueName: \"kubernetes.io/projected/783a6498-488a-4698-a324-9de03bf9929e-kube-api-access-pls9m\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:23:53.104618 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.104610    2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/783a6498-488a-4698-a324-9de03bf9929e-tls-cert\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:23:53.591964 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.591925    2566 generic.go:356] "Generic (PLEG): container finished" podID="783a6498-488a-4698-a324-9de03bf9929e" containerID="317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179" exitCode=0
Jul 09 22:23:53.592465 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.591992    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5f889966-58v6v"
Jul 09 22:23:53.592465 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.592012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f889966-58v6v" event={"ID":"783a6498-488a-4698-a324-9de03bf9929e","Type":"ContainerDied","Data":"317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179"}
Jul 09 22:23:53.592465 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.592056    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5f889966-58v6v" event={"ID":"783a6498-488a-4698-a324-9de03bf9929e","Type":"ContainerDied","Data":"8497736ea3d1d0d60eacf3aa2165ae590415f6bdf7160c9f342d71c02874bd85"}
Jul 09 22:23:53.592465 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.592078    2566 scope.go:117] "RemoveContainer" containerID="317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179"
Jul 09 22:23:53.607117 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.607096    2566 scope.go:117] "RemoveContainer" containerID="317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179"
Jul 09 22:23:53.607460 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:23:53.607438    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179\": container with ID starting with 317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179 not found: ID does not exist" containerID="317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179"
Jul 09 22:23:53.607551 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.607473    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179"} err="failed to get container status \"317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179\": rpc error: code = NotFound desc = could not find container \"317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179\": container with ID starting with 317aa4f66bd7dd65a637c21c22ba78152367b2f343ead5bd012489fa951d4179 not found: ID does not exist"
Jul 09 22:23:53.616968 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.616928    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5f889966-58v6v"]
Jul 09 22:23:53.618584 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.618560    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b5f889966-58v6v"]
Jul 09 22:23:53.679515 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:23:53.679472    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="783a6498-488a-4698-a324-9de03bf9929e" path="/var/lib/kubelet/pods/783a6498-488a-4698-a324-9de03bf9929e/volumes"
Jul 09 22:24:20.589885 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:24:20.589843    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:24:32.387464 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:24:32.387425    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:24:42.601262 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:24:42.601218    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:24:51.791300 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:24:51.791264    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:25:02.587670 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:25:02.587582    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:26:07.782857 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:26:07.782821    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:26:17.788766 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:26:17.788729    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:26:28.098168 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:26:28.098133    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:26:34.093551 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:26:34.093466    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:26:44.795100 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:26:44.795065    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:26:54.390650 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:26:54.390603    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:27:05.594455 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:27:05.594419    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:27:15.586432 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:27:15.586396    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:27:22.285604 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:27:22.285571    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:27:29.785098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:27:29.785062    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:27:40.791331 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:27:40.791299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:27:51.689887 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:27:51.689852    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:28:14.990575 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:28:14.990494    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:28:27.590207 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:28:27.590170    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:28:43.595258 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:28:43.595212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:28:53.212825 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:28:53.212790    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:29:02.891609 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:29:02.891564    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:29:12.597138 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:29:12.597098    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:29:22.498851 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:29:22.498809    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:31:10.984655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:31:10.984574    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:31:19.490744 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:31:19.490705    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:31:34.682588 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:31:34.682552    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:31:46.600913 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:31:46.600873    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:31:58.682650 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:31:58.682615    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:32:07.693125 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:32:07.693089    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:32:18.189356 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:32:18.189315    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:32:27.690534 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:32:27.690500    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:32:38.589308 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:32:38.589188    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:32:48.886898 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:32:48.886865    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:32:59.895625 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:32:59.895590    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:33:21.106232 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:33:21.106193    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:33:30.689937 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:33:30.689903    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:33:40.888094 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:33:40.888048    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:33:50.992685 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:33:50.992648    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:34:00.592129 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:34:00.592087    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:34:12.796481 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:34:12.796407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:34:26.996835 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:34:26.996797    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:34:36.426547 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:34:36.426515    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:34:50.286781 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:34:50.286736    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:34:59.894665 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:34:59.894630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:35:13.307718 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:35:13.307681    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:35:23.193008 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:35:23.192962    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:35:32.982381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:35:32.982299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:35:45.394653 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:35:45.394609    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:35:56.592358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:35:56.592323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:36:05.385289 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:36:05.385231    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:36:17.617147 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:36:17.617113    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:36:28.688577 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:36:28.688536    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:36:37.702787 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:36:37.702748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:36:52.201175 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:36:52.201138    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:37:00.999866 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:37:00.999829    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:37:10.699712 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:37:10.699637    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:37:18.247164 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:37:18.247126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:37:27.697479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:37:27.697391    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:37:47.107338 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:37:47.107300    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:37:58.884914 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:37:58.884875    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:39:19.896687 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:39:19.896656    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:39:29.289224 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:39:29.289177    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:39:41.245864 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:39:41.245825    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:39:52.188756 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:39:52.188724    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:40:22.691570 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:22.691534    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:40:45.925411 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.925375    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"]
Jul 09 22:40:45.925829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.925790    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="783a6498-488a-4698-a324-9de03bf9929e" containerName="authorino"
Jul 09 22:40:45.925829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.925804    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="783a6498-488a-4698-a324-9de03bf9929e" containerName="authorino"
Jul 09 22:40:45.925902 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.925865    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="783a6498-488a-4698-a324-9de03bf9929e" containerName="authorino"
Jul 09 22:40:45.928848 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.928823    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:45.932727 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.932708    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f040043d\"/\"kube-root-ca.crt\""
Jul 09 22:40:45.932866 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.932707    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f040043d\"/\"e2e-lifecycle-model-f040043d-kserve-self-signed-certs\""
Jul 09 22:40:45.933302 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.933284    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f040043d\"/\"openshift-service-ca.crt\""
Jul 09 22:40:45.933381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.933325    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f040043d\"/\"default-dockercfg-5kkld\""
Jul 09 22:40:45.946108 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.946080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"]
Jul 09 22:40:45.993208 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.993174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-model-cache\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:45.993208 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.993210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-home\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:45.993428 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.993232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tls-certs\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:45.993428 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.993288    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pqcd\" (UniqueName: \"kubernetes.io/projected/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-kube-api-access-5pqcd\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:45.993428 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.993373    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-dshm\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:45.993428 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:45.993406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tmp-dir\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094351 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094311    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pqcd\" (UniqueName: \"kubernetes.io/projected/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-kube-api-access-5pqcd\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094371    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-dshm\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tmp-dir\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-model-cache\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-home\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094491    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tls-certs\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094866 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094844    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tmp-dir\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094929 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094854    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-model-cache\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.094929 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.094904    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-home\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.096605 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.096584    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-dshm\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.097005 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.096984    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tls-certs\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.104941 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.104913    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pqcd\" (UniqueName: \"kubernetes.io/projected/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-kube-api-access-5pqcd\") pod \"e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") " pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.238577 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.238532    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:46.375050 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.375023    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"]
Jul 09 22:40:46.377721 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:40:46.377696    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e89d71c_5981_46f4_8dc0_7bf6cc79b481.slice/crio-8fef0870d6253e74b4e2daef6ddbc80a38ae418718f751ca66c076f6464a6ed8 WatchSource:0}: Error finding container 8fef0870d6253e74b4e2daef6ddbc80a38ae418718f751ca66c076f6464a6ed8: Status 404 returned error can't find the container with id 8fef0870d6253e74b4e2daef6ddbc80a38ae418718f751ca66c076f6464a6ed8
Jul 09 22:40:46.379674 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.379654    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 22:40:46.487482 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:46.487444    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm" event={"ID":"7e89d71c-5981-46f4-8dc0-7bf6cc79b481","Type":"ContainerStarted","Data":"8fef0870d6253e74b4e2daef6ddbc80a38ae418718f751ca66c076f6464a6ed8"}
Jul 09 22:40:48.501894 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:48.501859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm" event={"ID":"7e89d71c-5981-46f4-8dc0-7bf6cc79b481","Type":"ContainerStarted","Data":"f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0"}
Jul 09 22:40:48.520382 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:48.520331    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm" podStartSLOduration=2.404804743 podStartE2EDuration="3.520314479s" totalImagesPullingTime="1.115509736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:40:48.519490165 +0000 UTC m=+1928.410520681" watchObservedRunningTime="2026-07-09 22:40:48.520314479 +0000 UTC m=+1928.411344992"
Jul 09 22:40:49.491300 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:49.491269    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:40:56.238785 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:56.238743    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:56.239210 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:56.238815    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:56.251716 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:56.251689    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:40:56.553981 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:40:56.553899    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:41:01.695595 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:01.695553    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:41:15.356069 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:15.356031    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"]
Jul 09 22:41:15.501223 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:15.501188    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:41:30.376383 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.376335    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm" podUID="7e89d71c-5981-46f4-8dc0-7bf6cc79b481" containerName="main" containerID="cri-o://f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0" gracePeriod=45
Jul 09 22:41:30.627036 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.626961    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:41:30.699356 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-model-cache\") pod \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") "
Jul 09 22:41:30.699356 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699356    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-dshm\") pod \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") "
Jul 09 22:41:30.699590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699405    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tls-certs\") pod \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") "
Jul 09 22:41:30.699590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699448    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pqcd\" (UniqueName: \"kubernetes.io/projected/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-kube-api-access-5pqcd\") pod \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") "
Jul 09 22:41:30.699590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699476    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-home\") pod \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") "
Jul 09 22:41:30.699590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699510    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tmp-dir\") pod \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\" (UID: \"7e89d71c-5981-46f4-8dc0-7bf6cc79b481\") "
Jul 09 22:41:30.699794 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699632    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e89d71c-5981-46f4-8dc0-7bf6cc79b481" (UID: "7e89d71c-5981-46f4-8dc0-7bf6cc79b481"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:41:30.699843 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699819    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-home" (OuterVolumeSpecName: "home") pod "7e89d71c-5981-46f4-8dc0-7bf6cc79b481" (UID: "7e89d71c-5981-46f4-8dc0-7bf6cc79b481"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:41:30.699876 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699829    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e89d71c-5981-46f4-8dc0-7bf6cc79b481" (UID: "7e89d71c-5981-46f4-8dc0-7bf6cc79b481"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:41:30.699876 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.699860    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:41:30.701807 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.701782    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-dshm" (OuterVolumeSpecName: "dshm") pod "7e89d71c-5981-46f4-8dc0-7bf6cc79b481" (UID: "7e89d71c-5981-46f4-8dc0-7bf6cc79b481"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:41:30.701908 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.701819    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-kube-api-access-5pqcd" (OuterVolumeSpecName: "kube-api-access-5pqcd") pod "7e89d71c-5981-46f4-8dc0-7bf6cc79b481" (UID: "7e89d71c-5981-46f4-8dc0-7bf6cc79b481"). InnerVolumeSpecName "kube-api-access-5pqcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:41:30.701908 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.701882    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e89d71c-5981-46f4-8dc0-7bf6cc79b481" (UID: "7e89d71c-5981-46f4-8dc0-7bf6cc79b481"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:41:30.710670 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.710632    2566 generic.go:356] "Generic (PLEG): container finished" podID="7e89d71c-5981-46f4-8dc0-7bf6cc79b481" containerID="f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0" exitCode=0
Jul 09 22:41:30.710802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.710681    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"
Jul 09 22:41:30.710802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.710714    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm" event={"ID":"7e89d71c-5981-46f4-8dc0-7bf6cc79b481","Type":"ContainerDied","Data":"f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0"}
Jul 09 22:41:30.710802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.710761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm" event={"ID":"7e89d71c-5981-46f4-8dc0-7bf6cc79b481","Type":"ContainerDied","Data":"8fef0870d6253e74b4e2daef6ddbc80a38ae418718f751ca66c076f6464a6ed8"}
Jul 09 22:41:30.710802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.710783    2566 scope.go:117] "RemoveContainer" containerID="f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0"
Jul 09 22:41:30.723213 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.723188    2566 scope.go:117] "RemoveContainer" containerID="f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0"
Jul 09 22:41:30.723564 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:41:30.723542    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0\": container with ID starting with f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0 not found: ID does not exist" containerID="f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0"
Jul 09 22:41:30.723635 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.723576    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0"} err="failed to get container status \"f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0\": rpc error: code = NotFound desc = could not find container \"f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0\": container with ID starting with f38b33fd1ef98261fb71d2f59a59807014dee55271581b2b5c39e41ed5a047f0 not found: ID does not exist"
Jul 09 22:41:30.734153 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.734119    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"]
Jul 09 22:41:30.737899 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.737863    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f040043d/e2e-lifecycle-model-f040043d-kserve-5d8f9777dc-8k6fm"]
Jul 09 22:41:30.800341 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.800309    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:41:30.800341 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.800337    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pqcd\" (UniqueName: \"kubernetes.io/projected/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-kube-api-access-5pqcd\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:41:30.800341 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.800347    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:41:30.800569 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.800357    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:41:30.800569 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:30.800367    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e89d71c-5981-46f4-8dc0-7bf6cc79b481-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:41:31.679924 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:31.679894    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e89d71c-5981-46f4-8dc0-7bf6cc79b481" path="/var/lib/kubelet/pods/7e89d71c-5981-46f4-8dc0-7bf6cc79b481/volumes"
Jul 09 22:41:55.392083 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:41:55.392040    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:42:05.102134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:42:05.102093    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:42:16.023977 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:42:16.023942    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:43:05.290486 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:05.290405    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:43:19.588569 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:19.588533    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:43:31.097600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.097565    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"]
Jul 09 22:43:31.097981 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.097931    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e89d71c-5981-46f4-8dc0-7bf6cc79b481" containerName="main"
Jul 09 22:43:31.097981 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.097942    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e89d71c-5981-46f4-8dc0-7bf6cc79b481" containerName="main"
Jul 09 22:43:31.098052 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.098007    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e89d71c-5981-46f4-8dc0-7bf6cc79b481" containerName="main"
Jul 09 22:43:31.101014 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.100997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.104337 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.104307    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0295ed\"/\"openshift-service-ca.crt\""
Jul 09 22:43:31.104337 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.104309    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-0295ed\"/\"kube-root-ca.crt\""
Jul 09 22:43:31.104547 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.104310    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0295ed\"/\"test-model-0295ed-kserve-self-signed-certs\""
Jul 09 22:43:31.104547 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.104313    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-0295ed\"/\"default-dockercfg-bjl2x\""
Jul 09 22:43:31.110986 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.110966    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"]
Jul 09 22:43:31.276179 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.276143    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-model-cache\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.276398 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.276182    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxz7x\" (UniqueName: \"kubernetes.io/projected/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-kube-api-access-bxz7x\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.276398 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.276211    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-dshm\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.276398 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.276315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tls-certs\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.276398 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.276387    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tmp-dir\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.276546 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.276434    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-home\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.377882 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.377781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-model-cache\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.377882 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.377818    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxz7x\" (UniqueName: \"kubernetes.io/projected/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-kube-api-access-bxz7x\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.377882 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.377841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-dshm\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.377882 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.377863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tls-certs\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.378160 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.378007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tmp-dir\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.378160 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.378065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-home\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.378302 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.378281    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-model-cache\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.378371 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.378359    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tmp-dir\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.378427 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.378397    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-home\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.380192 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.380165    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-dshm\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.380497 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.380479    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tls-certs\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.386933 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.386904    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxz7x\" (UniqueName: \"kubernetes.io/projected/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-kube-api-access-bxz7x\") pod \"test-model-0295ed-kserve-66c88df4c5-xmfdq\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") " pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.411009 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.410974    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:31.547165 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:31.547114    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"]
Jul 09 22:43:31.551176 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:43:31.551146    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbecf9e1_daf0_4612_ab78_a5a40151c6b2.slice/crio-f0435c1c2949a312468bbc2b8dad802293ed892405636a6030d6c3bb0c3d3dfc WatchSource:0}: Error finding container f0435c1c2949a312468bbc2b8dad802293ed892405636a6030d6c3bb0c3d3dfc: Status 404 returned error can't find the container with id f0435c1c2949a312468bbc2b8dad802293ed892405636a6030d6c3bb0c3d3dfc
Jul 09 22:43:32.299151 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:32.299113    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq" event={"ID":"dbecf9e1-daf0-4612-ab78-a5a40151c6b2","Type":"ContainerStarted","Data":"e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86"}
Jul 09 22:43:32.299151 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:32.299160    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq" event={"ID":"dbecf9e1-daf0-4612-ab78-a5a40151c6b2","Type":"ContainerStarted","Data":"f0435c1c2949a312468bbc2b8dad802293ed892405636a6030d6c3bb0c3d3dfc"}
Jul 09 22:43:32.317369 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:32.317318    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq" podStartSLOduration=1.317303573 podStartE2EDuration="1.317303573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:43:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:43:32.315362443 +0000 UTC m=+2092.206392972" watchObservedRunningTime="2026-07-09 22:43:32.317303573 +0000 UTC m=+2092.208334086"
Jul 09 22:43:34.808918 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:34.808872    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:43:41.411718 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:41.411678    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:41.412130 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:41.411760    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:41.424633 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:41.424602    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:43:42.362976 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:43:42.362939    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:44:03.092414 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.092377    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"]
Jul 09 22:44:03.095790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.095771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.098338 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.098313    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c31bfc\"/\"openshift-service-ca.crt\""
Jul 09 22:44:03.099304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.099286    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c31bfc\"/\"test-model-c31bfc-kserve-self-signed-certs\""
Jul 09 22:44:03.099406 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.099313    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c31bfc\"/\"kube-root-ca.crt\""
Jul 09 22:44:03.099406 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.099340    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c31bfc\"/\"default-dockercfg-njrc4\""
Jul 09 22:44:03.109412 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.109384    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"]
Jul 09 22:44:03.167331 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.167286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-dshm\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.167331 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.167336    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-home\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.167548 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.167420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tmp-dir\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.167548 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.167448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tls-certs\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.167548 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.167469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftzsp\" (UniqueName: \"kubernetes.io/projected/e2990d89-ef42-463d-83d1-3a1b3238a4c5-kube-api-access-ftzsp\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.167648 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.167579    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-model-cache\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.268612 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.268561    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-model-cache\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.268960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.268667    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-dshm\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.268960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.268705    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-home\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.268960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.268753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tmp-dir\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.268960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.268778    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tls-certs\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.268960 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.268803    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftzsp\" (UniqueName: \"kubernetes.io/projected/e2990d89-ef42-463d-83d1-3a1b3238a4c5-kube-api-access-ftzsp\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.269240 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.269096    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-model-cache\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.269240 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.269151    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-home\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.269240 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.269226    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tmp-dir\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.271121 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.271090    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-dshm\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.271316 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.271299    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tls-certs\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.278819 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.278787    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftzsp\" (UniqueName: \"kubernetes.io/projected/e2990d89-ef42-463d-83d1-3a1b3238a4c5-kube-api-access-ftzsp\") pod \"test-model-c31bfc-kserve-84c6b7cd69-gpmjq\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") " pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.405762 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.405658    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:03.577888 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:03.577853    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"]
Jul 09 22:44:03.578229 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:44:03.578207    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2990d89_ef42_463d_83d1_3a1b3238a4c5.slice/crio-8965846d5df39caf8af35981d6630b85f87f501afeeb62055c56bde4b0d661d4 WatchSource:0}: Error finding container 8965846d5df39caf8af35981d6630b85f87f501afeeb62055c56bde4b0d661d4: Status 404 returned error can't find the container with id 8965846d5df39caf8af35981d6630b85f87f501afeeb62055c56bde4b0d661d4
Jul 09 22:44:04.468526 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:04.468487    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq" event={"ID":"e2990d89-ef42-463d-83d1-3a1b3238a4c5","Type":"ContainerStarted","Data":"247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c"}
Jul 09 22:44:04.468526 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:04.468532    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq" event={"ID":"e2990d89-ef42-463d-83d1-3a1b3238a4c5","Type":"ContainerStarted","Data":"8965846d5df39caf8af35981d6630b85f87f501afeeb62055c56bde4b0d661d4"}
Jul 09 22:44:04.488127 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:04.488071    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq" podStartSLOduration=1.488054506 podStartE2EDuration="1.488054506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:44:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:44:04.486638869 +0000 UTC m=+2124.377669388" watchObservedRunningTime="2026-07-09 22:44:04.488054506 +0000 UTC m=+2124.379085019"
Jul 09 22:44:07.197256 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:07.197218    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:44:13.406163 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:13.406111    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:13.406163 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:13.406178    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:13.419216 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:13.419189    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:13.525546 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:13.525510    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:21.889723 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:21.889688    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:44:24.228172 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:24.228132    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"]
Jul 09 22:44:25.214962 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:25.214929    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"]
Jul 09 22:44:38.111471 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:38.111389    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:44:39.248944 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.248885    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq" podUID="dbecf9e1-daf0-4612-ab78-a5a40151c6b2" containerName="main" containerID="cri-o://e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86" gracePeriod=45
Jul 09 22:44:39.502699 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.502644    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:44:39.597219 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597185    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxz7x\" (UniqueName: \"kubernetes.io/projected/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-kube-api-access-bxz7x\") pod \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") "
Jul 09 22:44:39.597219 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597224    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tls-certs\") pod \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") "
Jul 09 22:44:39.597491 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597267    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-home\") pod \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") "
Jul 09 22:44:39.597491 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597365    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tmp-dir\") pod \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") "
Jul 09 22:44:39.597491 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597481    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-model-cache\") pod \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") "
Jul 09 22:44:39.597491 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597487    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-home" (OuterVolumeSpecName: "home") pod "dbecf9e1-daf0-4612-ab78-a5a40151c6b2" (UID: "dbecf9e1-daf0-4612-ab78-a5a40151c6b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:39.597764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597509    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-dshm\") pod \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\" (UID: \"dbecf9e1-daf0-4612-ab78-a5a40151c6b2\") "
Jul 09 22:44:39.597764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597672    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "dbecf9e1-daf0-4612-ab78-a5a40151c6b2" (UID: "dbecf9e1-daf0-4612-ab78-a5a40151c6b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:39.597764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597685    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dbecf9e1-daf0-4612-ab78-a5a40151c6b2" (UID: "dbecf9e1-daf0-4612-ab78-a5a40151c6b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:39.597878 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597822    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:39.597878 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597840    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:39.597878 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.597852    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:39.599603 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.599582    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-kube-api-access-bxz7x" (OuterVolumeSpecName: "kube-api-access-bxz7x") pod "dbecf9e1-daf0-4612-ab78-a5a40151c6b2" (UID: "dbecf9e1-daf0-4612-ab78-a5a40151c6b2"). InnerVolumeSpecName "kube-api-access-bxz7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:44:39.599705 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.599634    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dbecf9e1-daf0-4612-ab78-a5a40151c6b2" (UID: "dbecf9e1-daf0-4612-ab78-a5a40151c6b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:44:39.599705 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.599633    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-dshm" (OuterVolumeSpecName: "dshm") pod "dbecf9e1-daf0-4612-ab78-a5a40151c6b2" (UID: "dbecf9e1-daf0-4612-ab78-a5a40151c6b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:39.641634 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.641607    2566 generic.go:356] "Generic (PLEG): container finished" podID="dbecf9e1-daf0-4612-ab78-a5a40151c6b2" containerID="e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86" exitCode=0
Jul 09 22:44:39.641751 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.641648    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"
Jul 09 22:44:39.641751 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.641682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq" event={"ID":"dbecf9e1-daf0-4612-ab78-a5a40151c6b2","Type":"ContainerDied","Data":"e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86"}
Jul 09 22:44:39.641751 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.641721    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq" event={"ID":"dbecf9e1-daf0-4612-ab78-a5a40151c6b2","Type":"ContainerDied","Data":"f0435c1c2949a312468bbc2b8dad802293ed892405636a6030d6c3bb0c3d3dfc"}
Jul 09 22:44:39.641751 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.641738    2566 scope.go:117] "RemoveContainer" containerID="e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86"
Jul 09 22:44:39.652892 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.652051    2566 scope.go:117] "RemoveContainer" containerID="e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86"
Jul 09 22:44:39.653096 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:44:39.653074    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86\": container with ID starting with e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86 not found: ID does not exist" containerID="e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86"
Jul 09 22:44:39.653190 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.653101    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86"} err="failed to get container status \"e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86\": rpc error: code = NotFound desc = could not find container \"e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86\": container with ID starting with e9803179188a156e49b8cb05ed1c857f39a67e6319df75613ac8c802ee368d86 not found: ID does not exist"
Jul 09 22:44:39.663475 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.663455    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"]
Jul 09 22:44:39.670981 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.670962    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-0295ed/test-model-0295ed-kserve-66c88df4c5-xmfdq"]
Jul 09 22:44:39.679053 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.679033    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbecf9e1-daf0-4612-ab78-a5a40151c6b2" path="/var/lib/kubelet/pods/dbecf9e1-daf0-4612-ab78-a5a40151c6b2/volumes"
Jul 09 22:44:39.698528 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.698511    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:39.698604 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.698530    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxz7x\" (UniqueName: \"kubernetes.io/projected/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-kube-api-access-bxz7x\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:39.698604 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:39.698540    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbecf9e1-daf0-4612-ab78-a5a40151c6b2-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:40.235205 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.235166    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq" podUID="e2990d89-ef42-463d-83d1-3a1b3238a4c5" containerName="main" containerID="cri-o://247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c" gracePeriod=45
Jul 09 22:44:40.488990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.488936    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:40.606529 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tls-certs\") pod \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") "
Jul 09 22:44:40.606708 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606538    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tmp-dir\") pod \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") "
Jul 09 22:44:40.606708 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606562    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftzsp\" (UniqueName: \"kubernetes.io/projected/e2990d89-ef42-463d-83d1-3a1b3238a4c5-kube-api-access-ftzsp\") pod \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") "
Jul 09 22:44:40.606708 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606591    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-home\") pod \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") "
Jul 09 22:44:40.606708 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606633    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-model-cache\") pod \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") "
Jul 09 22:44:40.606708 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606676    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-dshm\") pod \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\" (UID: \"e2990d89-ef42-463d-83d1-3a1b3238a4c5\") "
Jul 09 22:44:40.607340 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606841    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e2990d89-ef42-463d-83d1-3a1b3238a4c5" (UID: "e2990d89-ef42-463d-83d1-3a1b3238a4c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:40.607340 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606885    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-home" (OuterVolumeSpecName: "home") pod "e2990d89-ef42-463d-83d1-3a1b3238a4c5" (UID: "e2990d89-ef42-463d-83d1-3a1b3238a4c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:40.607340 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.606897    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e2990d89-ef42-463d-83d1-3a1b3238a4c5" (UID: "e2990d89-ef42-463d-83d1-3a1b3238a4c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:40.608830 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.608806    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e2990d89-ef42-463d-83d1-3a1b3238a4c5" (UID: "e2990d89-ef42-463d-83d1-3a1b3238a4c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:44:40.608830 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.608821    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-dshm" (OuterVolumeSpecName: "dshm") pod "e2990d89-ef42-463d-83d1-3a1b3238a4c5" (UID: "e2990d89-ef42-463d-83d1-3a1b3238a4c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:44:40.608968 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.608870    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2990d89-ef42-463d-83d1-3a1b3238a4c5-kube-api-access-ftzsp" (OuterVolumeSpecName: "kube-api-access-ftzsp") pod "e2990d89-ef42-463d-83d1-3a1b3238a4c5" (UID: "e2990d89-ef42-463d-83d1-3a1b3238a4c5"). InnerVolumeSpecName "kube-api-access-ftzsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:44:40.648309 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.648281    2566 generic.go:356] "Generic (PLEG): container finished" podID="e2990d89-ef42-463d-83d1-3a1b3238a4c5" containerID="247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c" exitCode=0
Jul 09 22:44:40.648429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.648318    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"
Jul 09 22:44:40.648429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.648328    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq" event={"ID":"e2990d89-ef42-463d-83d1-3a1b3238a4c5","Type":"ContainerDied","Data":"247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c"}
Jul 09 22:44:40.648429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.648358    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq" event={"ID":"e2990d89-ef42-463d-83d1-3a1b3238a4c5","Type":"ContainerDied","Data":"8965846d5df39caf8af35981d6630b85f87f501afeeb62055c56bde4b0d661d4"}
Jul 09 22:44:40.648429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.648374    2566 scope.go:117] "RemoveContainer" containerID="247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c"
Jul 09 22:44:40.656992 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.656972    2566 scope.go:117] "RemoveContainer" containerID="247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c"
Jul 09 22:44:40.657218 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:44:40.657197    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c\": container with ID starting with 247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c not found: ID does not exist" containerID="247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c"
Jul 09 22:44:40.657304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.657226    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c"} err="failed to get container status \"247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c\": rpc error: code = NotFound desc = could not find container \"247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c\": container with ID starting with 247783a5dda5a6f2324b381f5c9e35df30a3b9d4218fa6a27529c298c7dc9b4c not found: ID does not exist"
Jul 09 22:44:40.677109 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.677087    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"]
Jul 09 22:44:40.694928 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.694904    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c31bfc/test-model-c31bfc-kserve-84c6b7cd69-gpmjq"]
Jul 09 22:44:40.707646 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.707621    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:40.707646 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.707643    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftzsp\" (UniqueName: \"kubernetes.io/projected/e2990d89-ef42-463d-83d1-3a1b3238a4c5-kube-api-access-ftzsp\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:40.707769 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.707653    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:40.707769 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.707663    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:40.707769 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.707672    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2990d89-ef42-463d-83d1-3a1b3238a4c5-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:40.707769 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:40.707680    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2990d89-ef42-463d-83d1-3a1b3238a4c5-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:44:41.679832 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:44:41.679801    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2990d89-ef42-463d-83d1-3a1b3238a4c5" path="/var/lib/kubelet/pods/e2990d89-ef42-463d-83d1-3a1b3238a4c5/volumes"
Jul 09 22:45:00.155539 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.155497    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"]
Jul 09 22:45:00.156037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.155912    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbecf9e1-daf0-4612-ab78-a5a40151c6b2" containerName="main"
Jul 09 22:45:00.156037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.155926    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbecf9e1-daf0-4612-ab78-a5a40151c6b2" containerName="main"
Jul 09 22:45:00.156037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.155937    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2990d89-ef42-463d-83d1-3a1b3238a4c5" containerName="main"
Jul 09 22:45:00.156037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.155943    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2990d89-ef42-463d-83d1-3a1b3238a4c5" containerName="main"
Jul 09 22:45:00.156037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.156002    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2990d89-ef42-463d-83d1-3a1b3238a4c5" containerName="main"
Jul 09 22:45:00.156037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.156013    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbecf9e1-daf0-4612-ab78-a5a40151c6b2" containerName="main"
Jul 09 22:45:00.158772 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.158752    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"]
Jul 09 22:45:00.158910 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.158884    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:45:00.161860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.161841    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:00.161964 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.161915    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 22:45:00.162003 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.161979    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 22:45:00.164044 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.164028    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-57hq5\""
Jul 09 22:45:00.170967 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.170948    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"]
Jul 09 22:45:00.177217 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.177193    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"]
Jul 09 22:45:00.285783 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.285745    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss7th\" (UniqueName: \"kubernetes.io/projected/5f86ac90-3ff9-4f9b-861f-b97b01053bdb-kube-api-access-ss7th\") pod \"maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph\" (UID: \"5f86ac90-3ff9-4f9b-861f-b97b01053bdb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:00.285990 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.285796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccmw8\" (UniqueName: \"kubernetes.io/projected/379f5f71-598c-41d7-adb5-6fb899f1aec1-kube-api-access-ccmw8\") pod \"maas-api-key-cleanup-29727285-h4rr9\" (UID: \"379f5f71-598c-41d7-adb5-6fb899f1aec1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:45:00.387358 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.387320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss7th\" (UniqueName: \"kubernetes.io/projected/5f86ac90-3ff9-4f9b-861f-b97b01053bdb-kube-api-access-ss7th\") pod \"maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph\" (UID: \"5f86ac90-3ff9-4f9b-861f-b97b01053bdb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:00.387556 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.387379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccmw8\" (UniqueName: \"kubernetes.io/projected/379f5f71-598c-41d7-adb5-6fb899f1aec1-kube-api-access-ccmw8\") pod \"maas-api-key-cleanup-29727285-h4rr9\" (UID: \"379f5f71-598c-41d7-adb5-6fb899f1aec1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:45:00.397384 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.397357    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccmw8\" (UniqueName: \"kubernetes.io/projected/379f5f71-598c-41d7-adb5-6fb899f1aec1-kube-api-access-ccmw8\") pod \"maas-api-key-cleanup-29727285-h4rr9\" (UID: \"379f5f71-598c-41d7-adb5-6fb899f1aec1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:45:00.398672 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.398651    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss7th\" (UniqueName: \"kubernetes.io/projected/5f86ac90-3ff9-4f9b-861f-b97b01053bdb-kube-api-access-ss7th\") pod \"maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph\" (UID: \"5f86ac90-3ff9-4f9b-861f-b97b01053bdb\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:00.471622 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.471587    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:45:00.479278 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.479241    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:00.619804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.619776    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"]
Jul 09 22:45:00.622326 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:45:00.622298    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379f5f71_598c_41d7_adb5_6fb899f1aec1.slice/crio-b0dda88053f69b252ad329992f23a5eafb462b70dde6ecc63b8ad6a894814ac0 WatchSource:0}: Error finding container b0dda88053f69b252ad329992f23a5eafb462b70dde6ecc63b8ad6a894814ac0: Status 404 returned error can't find the container with id b0dda88053f69b252ad329992f23a5eafb462b70dde6ecc63b8ad6a894814ac0
Jul 09 22:45:00.637620 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.637552    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"]
Jul 09 22:45:00.641236 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:45:00.641214    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f86ac90_3ff9_4f9b_861f_b97b01053bdb.slice/crio-836e7d47819c81a2aef452d58a8f4996f46298f54e73c8fbcb29f95218179ed2 WatchSource:0}: Error finding container 836e7d47819c81a2aef452d58a8f4996f46298f54e73c8fbcb29f95218179ed2: Status 404 returned error can't find the container with id 836e7d47819c81a2aef452d58a8f4996f46298f54e73c8fbcb29f95218179ed2
Jul 09 22:45:00.752711 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.752605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph" event={"ID":"5f86ac90-3ff9-4f9b-861f-b97b01053bdb","Type":"ContainerStarted","Data":"836e7d47819c81a2aef452d58a8f4996f46298f54e73c8fbcb29f95218179ed2"}
Jul 09 22:45:00.753797 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:00.753769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9" event={"ID":"379f5f71-598c-41d7-adb5-6fb899f1aec1","Type":"ContainerStarted","Data":"b0dda88053f69b252ad329992f23a5eafb462b70dde6ecc63b8ad6a894814ac0"}
Jul 09 22:45:03.771903 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:03.771868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9" event={"ID":"379f5f71-598c-41d7-adb5-6fb899f1aec1","Type":"ContainerStarted","Data":"96596f3ed39f1059ccc0025a846724db55f07e4b6f50401b75662130553395ba"}
Jul 09 22:45:03.773453 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:03.773427    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph" event={"ID":"5f86ac90-3ff9-4f9b-861f-b97b01053bdb","Type":"ContainerStarted","Data":"d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de"}
Jul 09 22:45:03.789860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:03.789803    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9" podStartSLOduration=1.496089235 podStartE2EDuration="3.789786726s" totalImagesPullingTime="2.293697491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:45:03.787859819 +0000 UTC m=+2183.678890345" watchObservedRunningTime="2026-07-09 22:45:03.789786726 +0000 UTC m=+2183.680817272"
Jul 09 22:45:03.803444 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:03.803391    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph" podStartSLOduration=1.532667334 podStartE2EDuration="3.803374155s" totalImagesPullingTime="2.270706821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:45:03.802181234 +0000 UTC m=+2183.693211747" watchObservedRunningTime="2026-07-09 22:45:03.803374155 +0000 UTC m=+2183.694404668"
Jul 09 22:45:09.904093 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:09.904057    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"]
Jul 09 22:45:09.904653 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:09.904281    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph" podUID="5f86ac90-3ff9-4f9b-861f-b97b01053bdb" containerName="cleanup" containerID="cri-o://d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de" gracePeriod=30
Jul 09 22:45:12.693719 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:12.693686    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:45:27.289075 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:27.289037    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:45:40.092936 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.092908    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:40.152574 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.152533    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss7th\" (UniqueName: \"kubernetes.io/projected/5f86ac90-3ff9-4f9b-861f-b97b01053bdb-kube-api-access-ss7th\") pod \"5f86ac90-3ff9-4f9b-861f-b97b01053bdb\" (UID: \"5f86ac90-3ff9-4f9b-861f-b97b01053bdb\") "
Jul 09 22:45:40.154699 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.154673    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f86ac90-3ff9-4f9b-861f-b97b01053bdb-kube-api-access-ss7th" (OuterVolumeSpecName: "kube-api-access-ss7th") pod "5f86ac90-3ff9-4f9b-861f-b97b01053bdb" (UID: "5f86ac90-3ff9-4f9b-861f-b97b01053bdb"). InnerVolumeSpecName "kube-api-access-ss7th". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:45:40.253307 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.253275    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss7th\" (UniqueName: \"kubernetes.io/projected/5f86ac90-3ff9-4f9b-861f-b97b01053bdb-kube-api-access-ss7th\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:45:40.956297 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.956235    2566 generic.go:356] "Generic (PLEG): container finished" podID="5f86ac90-3ff9-4f9b-861f-b97b01053bdb" containerID="d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de" exitCode=137
Jul 09 22:45:40.956297 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.956281    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph" event={"ID":"5f86ac90-3ff9-4f9b-861f-b97b01053bdb","Type":"ContainerDied","Data":"d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de"}
Jul 09 22:45:40.956297 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.956299    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"
Jul 09 22:45:40.956699 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.956337    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph" event={"ID":"5f86ac90-3ff9-4f9b-861f-b97b01053bdb","Type":"ContainerDied","Data":"836e7d47819c81a2aef452d58a8f4996f46298f54e73c8fbcb29f95218179ed2"}
Jul 09 22:45:40.956699 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.956365    2566 scope.go:117] "RemoveContainer" containerID="d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de"
Jul 09 22:45:40.964999 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.964984    2566 scope.go:117] "RemoveContainer" containerID="d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de"
Jul 09 22:45:40.965217 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:45:40.965199    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de\": container with ID starting with d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de not found: ID does not exist" containerID="d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de"
Jul 09 22:45:40.965306 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.965226    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de"} err="failed to get container status \"d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de\": rpc error: code = NotFound desc = could not find container \"d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de\": container with ID starting with d2d6bce850731eded7c2a16209763fa0e1ff885a5c155f2d1cb17c99d09587de not found: ID does not exist"
Jul 09 22:45:40.978058 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.978037    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"]
Jul 09 22:45:40.985848 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:40.985827    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c31bfc-29727285-dbwph"]
Jul 09 22:45:41.680192 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:41.680159    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f86ac90-3ff9-4f9b-861f-b97b01053bdb" path="/var/lib/kubelet/pods/5f86ac90-3ff9-4f9b-861f-b97b01053bdb/volumes"
Jul 09 22:45:44.496610 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:44.496578    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:45:57.490905 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:45:57.490856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:46:00.020479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:00.020443    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"]
Jul 09 22:46:00.020956 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:00.020643    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9" podUID="379f5f71-598c-41d7-adb5-6fb899f1aec1" containerName="cleanup" containerID="cri-o://96596f3ed39f1059ccc0025a846724db55f07e4b6f50401b75662130553395ba" gracePeriod=30
Jul 09 22:46:09.898743 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:09.898667    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:46:22.903762 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.903731    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"]
Jul 09 22:46:22.904260 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.904098    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f86ac90-3ff9-4f9b-861f-b97b01053bdb" containerName="cleanup"
Jul 09 22:46:22.904260 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.904109    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f86ac90-3ff9-4f9b-861f-b97b01053bdb" containerName="cleanup"
Jul 09 22:46:22.904260 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.904168    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f86ac90-3ff9-4f9b-861f-b97b01053bdb" containerName="cleanup"
Jul 09 22:46:22.907073 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.907058    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:22.909736 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.909694    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a70a31\"/\"openshift-service-ca.crt\""
Jul 09 22:46:22.910519 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.910503    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a70a31\"/\"default-dockercfg-d8whm\""
Jul 09 22:46:22.910602 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.910584    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a70a31\"/\"auth-test-model-a70a31-kserve-self-signed-certs\""
Jul 09 22:46:22.910663 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.910619    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a70a31\"/\"kube-root-ca.crt\""
Jul 09 22:46:22.919423 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:22.919404    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"]
Jul 09 22:46:23.054362 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.054322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-home\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.054559 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.054375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tls-certs\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.054559 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.054404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-model-cache\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.054559 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.054442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42cd\" (UniqueName: \"kubernetes.io/projected/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-kube-api-access-f42cd\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.054559 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.054473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-dshm\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.054559 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.054525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tmp-dir\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155149 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155061    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-home\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155149 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tls-certs\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155353 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-model-cache\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155353 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155277    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f42cd\" (UniqueName: \"kubernetes.io/projected/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-kube-api-access-f42cd\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155353 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-dshm\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155496 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tmp-dir\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155496 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155477    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-home\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155553    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-model-cache\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.155749 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.155732    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tmp-dir\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.157516 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.157496    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-dshm\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.157672 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.157655    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tls-certs\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.163964 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.163943    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42cd\" (UniqueName: \"kubernetes.io/projected/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-kube-api-access-f42cd\") pod \"auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") " pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.216451 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.216421    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:23.348067 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:23.348036    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"]
Jul 09 22:46:23.349581 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:46:23.349546    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod312dd62e_a4f3_4ac8_9300_ddfcd8f659bb.slice/crio-ec33a44b58a3b17b4088916222b8e3161142a4ca6438727cbf917e8a70ee4646 WatchSource:0}: Error finding container ec33a44b58a3b17b4088916222b8e3161142a4ca6438727cbf917e8a70ee4646: Status 404 returned error can't find the container with id ec33a44b58a3b17b4088916222b8e3161142a4ca6438727cbf917e8a70ee4646
Jul 09 22:46:24.161011 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:24.160973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh" event={"ID":"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb","Type":"ContainerStarted","Data":"04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da"}
Jul 09 22:46:24.161438 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:24.161024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh" event={"ID":"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb","Type":"ContainerStarted","Data":"ec33a44b58a3b17b4088916222b8e3161142a4ca6438727cbf917e8a70ee4646"}
Jul 09 22:46:25.625586 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:25.625533    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh" podStartSLOduration=3.6255150069999997 podStartE2EDuration="3.625515007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:46:24.188976601 +0000 UTC m=+2264.080007114" watchObservedRunningTime="2026-07-09 22:46:25.625515007 +0000 UTC m=+2265.516545520"
Jul 09 22:46:25.626606 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:25.626588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:46:30.191513 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.191383    2566 generic.go:356] "Generic (PLEG): container finished" podID="379f5f71-598c-41d7-adb5-6fb899f1aec1" containerID="96596f3ed39f1059ccc0025a846724db55f07e4b6f50401b75662130553395ba" exitCode=137
Jul 09 22:46:30.191513 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.191450    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9" event={"ID":"379f5f71-598c-41d7-adb5-6fb899f1aec1","Type":"ContainerDied","Data":"96596f3ed39f1059ccc0025a846724db55f07e4b6f50401b75662130553395ba"}
Jul 09 22:46:30.191513 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.191502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9" event={"ID":"379f5f71-598c-41d7-adb5-6fb899f1aec1","Type":"ContainerDied","Data":"b0dda88053f69b252ad329992f23a5eafb462b70dde6ecc63b8ad6a894814ac0"}
Jul 09 22:46:30.191867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.191518    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0dda88053f69b252ad329992f23a5eafb462b70dde6ecc63b8ad6a894814ac0"
Jul 09 22:46:30.206518 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.206491    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:46:30.321595 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.321517    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccmw8\" (UniqueName: \"kubernetes.io/projected/379f5f71-598c-41d7-adb5-6fb899f1aec1-kube-api-access-ccmw8\") pod \"379f5f71-598c-41d7-adb5-6fb899f1aec1\" (UID: \"379f5f71-598c-41d7-adb5-6fb899f1aec1\") "
Jul 09 22:46:30.323538 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.323513    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379f5f71-598c-41d7-adb5-6fb899f1aec1-kube-api-access-ccmw8" (OuterVolumeSpecName: "kube-api-access-ccmw8") pod "379f5f71-598c-41d7-adb5-6fb899f1aec1" (UID: "379f5f71-598c-41d7-adb5-6fb899f1aec1"). InnerVolumeSpecName "kube-api-access-ccmw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:46:30.423152 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:30.423097    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccmw8\" (UniqueName: \"kubernetes.io/projected/379f5f71-598c-41d7-adb5-6fb899f1aec1-kube-api-access-ccmw8\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:46:31.196071 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:31.196041    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"
Jul 09 22:46:31.238482 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:31.238422    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"]
Jul 09 22:46:31.245098 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:31.245067    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29727285-h4rr9"]
Jul 09 22:46:31.680953 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:31.680924    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379f5f71-598c-41d7-adb5-6fb899f1aec1" path="/var/lib/kubelet/pods/379f5f71-598c-41d7-adb5-6fb899f1aec1/volumes"
Jul 09 22:46:33.217573 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:33.217532    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:33.217989 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:33.217683    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:33.230030 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:33.230001    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:34.224802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:34.224774    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:46:41.834062 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.834025    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"]
Jul 09 22:46:41.834470 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.834444    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379f5f71-598c-41d7-adb5-6fb899f1aec1" containerName="cleanup"
Jul 09 22:46:41.834470 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.834457    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="379f5f71-598c-41d7-adb5-6fb899f1aec1" containerName="cleanup"
Jul 09 22:46:41.834554 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.834530    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="379f5f71-598c-41d7-adb5-6fb899f1aec1" containerName="cleanup"
Jul 09 22:46:41.837847 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.837827    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:41.840300 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.840274    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"openshift-service-ca.crt\""
Jul 09 22:46:41.840705 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.840688    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"kube-root-ca.crt\""
Jul 09 22:46:41.841114 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.841095    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"default-dockercfg-bbv9x\""
Jul 09 22:46:41.841214 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.841176    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"auth-test-model-d3fb60-kserve-self-signed-certs\""
Jul 09 22:46:41.860314 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.860282    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"]
Jul 09 22:46:41.929597 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.929551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tls-certs\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:41.929839 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.929617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-home\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:41.929839 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.929681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-dshm\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:41.929839 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.929750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tmp-dir\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:41.929839 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.929796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d4pn\" (UniqueName: \"kubernetes.io/projected/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-kube-api-access-4d4pn\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:41.930020 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:41.929865    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-model-cache\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031214 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-home\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031214 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031210    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-dshm\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031461 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031239    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tmp-dir\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031461 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d4pn\" (UniqueName: \"kubernetes.io/projected/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-kube-api-access-4d4pn\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031527 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031466    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-model-cache\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031569 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031523    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tls-certs\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031645 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031625    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tmp-dir\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031721 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031699    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-home\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.031773 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.031749    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-model-cache\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.033655 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.033637    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-dshm\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.033796 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.033779    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tls-certs\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.041169 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.041132    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d4pn\" (UniqueName: \"kubernetes.io/projected/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-kube-api-access-4d4pn\") pod \"auth-test-model-d3fb60-kserve-bc75b867-dl8fb\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") " pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.148497 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.148402    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:42.303374 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:42.303349    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"]
Jul 09 22:46:42.305723 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:46:42.305686    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb260fb9_585e_4dd7_ae89_4626bc1e1a06.slice/crio-00a8a873ecc10eb9d623aac586ecf6fe69d22c41c50117f15450866d86fa7288 WatchSource:0}: Error finding container 00a8a873ecc10eb9d623aac586ecf6fe69d22c41c50117f15450866d86fa7288: Status 404 returned error can't find the container with id 00a8a873ecc10eb9d623aac586ecf6fe69d22c41c50117f15450866d86fa7288
Jul 09 22:46:43.260866 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:43.260834    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb" event={"ID":"fb260fb9-585e-4dd7-ae89-4626bc1e1a06","Type":"ContainerStarted","Data":"af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca"}
Jul 09 22:46:43.260866 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:43.260872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb" event={"ID":"fb260fb9-585e-4dd7-ae89-4626bc1e1a06","Type":"ContainerStarted","Data":"00a8a873ecc10eb9d623aac586ecf6fe69d22c41c50117f15450866d86fa7288"}
Jul 09 22:46:43.285674 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:43.285621    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb" podStartSLOduration=2.28560671 podStartE2EDuration="2.28560671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:46:43.284796173 +0000 UTC m=+2283.175826688" watchObservedRunningTime="2026-07-09 22:46:43.28560671 +0000 UTC m=+2283.176637237"
Jul 09 22:46:45.215554 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:45.215516    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:46:52.149121 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:52.149078    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:52.149629 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:52.149139    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:52.161421 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:52.161400    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:52.317965 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:52.317934    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:46:55.949463 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:46:55.949421    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:47:14.693796 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:47:14.693760    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:47:32.286746 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:47:32.286670    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:47:49.386943 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:47:49.386908    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:48:07.404790 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:48:07.404741    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:48:28.000345 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:48:28.000307    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:48:47.589126 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:48:47.589092    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:48:51.707770 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:48:51.707730    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"]
Jul 09 22:48:52.674795 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:48:52.674757    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"]
Jul 09 22:49:03.988686 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:03.988653    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:49:06.312804 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.312770    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"]
Jul 09 22:49:06.316475 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.316451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.318977 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.318957    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a70a31\"/\"sub-test-model-a70a31-kserve-self-signed-certs\""
Jul 09 22:49:06.329000 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.328977    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"]
Jul 09 22:49:06.359981 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.359957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-dshm\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.360119 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.359985    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-tmp-dir\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.360119 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.360005    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwz5\" (UniqueName: \"kubernetes.io/projected/5f8239b9-94e2-4914-8173-c666a2826066-kube-api-access-gzwz5\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.360119 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.360031    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f8239b9-94e2-4914-8173-c666a2826066-tls-certs\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.360119 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.360110    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-model-cache\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.360277 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.360128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-home\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-dshm\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-tmp-dir\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461665 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwz5\" (UniqueName: \"kubernetes.io/projected/5f8239b9-94e2-4914-8173-c666a2826066-kube-api-access-gzwz5\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461665 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f8239b9-94e2-4914-8173-c666a2826066-tls-certs\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461665 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-model-cache\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461828 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461651    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-home\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461884 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461865    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-tmp-dir\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.461936 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.461881    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-model-cache\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.462029 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.462001    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-home\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.463698 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.463678    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-dshm\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.463969 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.463952    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f8239b9-94e2-4914-8173-c666a2826066-tls-certs\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.471775 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.471749    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwz5\" (UniqueName: \"kubernetes.io/projected/5f8239b9-94e2-4914-8173-c666a2826066-kube-api-access-gzwz5\") pod \"sub-test-model-a70a31-kserve-64477bdbf7-g4q2s\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") " pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.625980 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.625885    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:06.728368 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.728321    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh" podUID="312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" containerName="main" containerID="cri-o://04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da" gracePeriod=45
Jul 09 22:49:06.773827 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.773802    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"]
Jul 09 22:49:06.797117 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:49:06.797080    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f8239b9_94e2_4914_8173_c666a2826066.slice/crio-e35edfe99d7654c29a2c632bc757d02a950f77a90a29c80fd03453ed87ae6fd6 WatchSource:0}: Error finding container e35edfe99d7654c29a2c632bc757d02a950f77a90a29c80fd03453ed87ae6fd6: Status 404 returned error can't find the container with id e35edfe99d7654c29a2c632bc757d02a950f77a90a29c80fd03453ed87ae6fd6
Jul 09 22:49:06.978355 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.978320    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:49:06.980676 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.980650    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s" event={"ID":"5f8239b9-94e2-4914-8173-c666a2826066","Type":"ContainerStarted","Data":"d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a"}
Jul 09 22:49:06.980810 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.980691    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s" event={"ID":"5f8239b9-94e2-4914-8173-c666a2826066","Type":"ContainerStarted","Data":"e35edfe99d7654c29a2c632bc757d02a950f77a90a29c80fd03453ed87ae6fd6"}
Jul 09 22:49:06.981918 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.981894    2566 generic.go:356] "Generic (PLEG): container finished" podID="312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" containerID="04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da" exitCode=0
Jul 09 22:49:06.982008 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.981967    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh" event={"ID":"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb","Type":"ContainerDied","Data":"04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da"}
Jul 09 22:49:06.984271 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.982299    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"
Jul 09 22:49:06.984271 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.982332    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh" event={"ID":"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb","Type":"ContainerDied","Data":"ec33a44b58a3b17b4088916222b8e3161142a4ca6438727cbf917e8a70ee4646"}
Jul 09 22:49:06.984271 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.982359    2566 scope.go:117] "RemoveContainer" containerID="04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da"
Jul 09 22:49:06.997492 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.997443    2566 scope.go:117] "RemoveContainer" containerID="04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da"
Jul 09 22:49:06.997798 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:49:06.997779    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da\": container with ID starting with 04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da not found: ID does not exist" containerID="04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da"
Jul 09 22:49:06.997886 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:06.997806    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da"} err="failed to get container status \"04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da\": rpc error: code = NotFound desc = could not find container \"04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da\": container with ID starting with 04c67d2ce477dd78af97d8e9b2b5c2f43a78c3ee792033e088ded53073c229da not found: ID does not exist"
Jul 09 22:49:07.016078 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.016018    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s" podStartSLOduration=1.015997961 podStartE2EDuration="1.015997961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:49:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:49:07.015104539 +0000 UTC m=+2426.906135035" watchObservedRunningTime="2026-07-09 22:49:07.015997961 +0000 UTC m=+2426.907028478"
Jul 09 22:49:07.066542 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066506    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tmp-dir\") pod \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") "
Jul 09 22:49:07.066711 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066597    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-dshm\") pod \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") "
Jul 09 22:49:07.066711 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066624    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-model-cache\") pod \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") "
Jul 09 22:49:07.066711 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066650    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-home\") pod \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") "
Jul 09 22:49:07.066711 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066677    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tls-certs\") pod \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") "
Jul 09 22:49:07.066711 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066701    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f42cd\" (UniqueName: \"kubernetes.io/projected/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-kube-api-access-f42cd\") pod \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\" (UID: \"312dd62e-a4f3-4ac8-9300-ddfcd8f659bb\") "
Jul 09 22:49:07.066945 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066800    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" (UID: "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:07.066945 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066856    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" (UID: "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:07.066945 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.066902    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-home" (OuterVolumeSpecName: "home") pod "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" (UID: "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:07.067104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.067062    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:07.067104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.067080    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:07.067104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.067104    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:07.068962 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.068937    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-dshm" (OuterVolumeSpecName: "dshm") pod "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" (UID: "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:07.069073 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.068996    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-kube-api-access-f42cd" (OuterVolumeSpecName: "kube-api-access-f42cd") pod "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" (UID: "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb"). InnerVolumeSpecName "kube-api-access-f42cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:49:07.069073 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.068994    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" (UID: "312dd62e-a4f3-4ac8-9300-ddfcd8f659bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:49:07.167789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.167717    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:07.167789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.167744    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:07.167789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.167755    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f42cd\" (UniqueName: \"kubernetes.io/projected/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb-kube-api-access-f42cd\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:07.303732 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.303699    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"]
Jul 09 22:49:07.307570 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.307548    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/auth-test-model-a70a31-kserve-6fb4874fb7-bwsbh"]
Jul 09 22:49:07.681528 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.681491    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" path="/var/lib/kubelet/pods/312dd62e-a4f3-4ac8-9300-ddfcd8f659bb/volumes"
Jul 09 22:49:07.695416 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.695379    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb" podUID="fb260fb9-585e-4dd7-ae89-4626bc1e1a06" containerName="main" containerID="cri-o://af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca" gracePeriod=45
Jul 09 22:49:07.950093 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.950060    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:49:07.990548 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.990512    2566 generic.go:356] "Generic (PLEG): container finished" podID="fb260fb9-585e-4dd7-ae89-4626bc1e1a06" containerID="af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca" exitCode=0
Jul 09 22:49:07.990732 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.990569    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"
Jul 09 22:49:07.990732 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.990603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb" event={"ID":"fb260fb9-585e-4dd7-ae89-4626bc1e1a06","Type":"ContainerDied","Data":"af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca"}
Jul 09 22:49:07.990732 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.990648    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb" event={"ID":"fb260fb9-585e-4dd7-ae89-4626bc1e1a06","Type":"ContainerDied","Data":"00a8a873ecc10eb9d623aac586ecf6fe69d22c41c50117f15450866d86fa7288"}
Jul 09 22:49:07.990732 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.990669    2566 scope.go:117] "RemoveContainer" containerID="af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca"
Jul 09 22:49:07.999585 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.999563    2566 scope.go:117] "RemoveContainer" containerID="af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca"
Jul 09 22:49:07.999892 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:49:07.999873    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca\": container with ID starting with af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca not found: ID does not exist" containerID="af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca"
Jul 09 22:49:07.999940 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:07.999904    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca"} err="failed to get container status \"af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca\": rpc error: code = NotFound desc = could not find container \"af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca\": container with ID starting with af603297ffdc10ec1cb7ab240bdbc2755de15468d71fd4db9d399f4c910b5dca not found: ID does not exist"
Jul 09 22:49:08.076890 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.076818    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-dshm\") pod \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") "
Jul 09 22:49:08.076890 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.076860    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-model-cache\") pod \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") "
Jul 09 22:49:08.077109 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.076887    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d4pn\" (UniqueName: \"kubernetes.io/projected/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-kube-api-access-4d4pn\") pod \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") "
Jul 09 22:49:08.077109 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.076928    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-home\") pod \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") "
Jul 09 22:49:08.077109 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.076968    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tmp-dir\") pod \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") "
Jul 09 22:49:08.077109 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077038    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tls-certs\") pod \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\" (UID: \"fb260fb9-585e-4dd7-ae89-4626bc1e1a06\") "
Jul 09 22:49:08.077332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077104    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb260fb9-585e-4dd7-ae89-4626bc1e1a06" (UID: "fb260fb9-585e-4dd7-ae89-4626bc1e1a06"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:08.077332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077200    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-home" (OuterVolumeSpecName: "home") pod "fb260fb9-585e-4dd7-ae89-4626bc1e1a06" (UID: "fb260fb9-585e-4dd7-ae89-4626bc1e1a06"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:08.077332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077289    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb260fb9-585e-4dd7-ae89-4626bc1e1a06" (UID: "fb260fb9-585e-4dd7-ae89-4626bc1e1a06"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:08.077854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077498    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:08.077854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077520    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:08.077854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.077534    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:08.079171 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.079148    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-kube-api-access-4d4pn" (OuterVolumeSpecName: "kube-api-access-4d4pn") pod "fb260fb9-585e-4dd7-ae89-4626bc1e1a06" (UID: "fb260fb9-585e-4dd7-ae89-4626bc1e1a06"). InnerVolumeSpecName "kube-api-access-4d4pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:49:08.079291 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.079179    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-dshm" (OuterVolumeSpecName: "dshm") pod "fb260fb9-585e-4dd7-ae89-4626bc1e1a06" (UID: "fb260fb9-585e-4dd7-ae89-4626bc1e1a06"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:49:08.079291 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.079227    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb260fb9-585e-4dd7-ae89-4626bc1e1a06" (UID: "fb260fb9-585e-4dd7-ae89-4626bc1e1a06"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:49:08.178469 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.178433    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:08.178469 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.178462    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:08.178469 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.178472    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d4pn\" (UniqueName: \"kubernetes.io/projected/fb260fb9-585e-4dd7-ae89-4626bc1e1a06-kube-api-access-4d4pn\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:49:08.312961 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.312927    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"]
Jul 09 22:49:08.317691 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:08.317666    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/auth-test-model-d3fb60-kserve-bc75b867-dl8fb"]
Jul 09 22:49:09.680429 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:09.680396    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb260fb9-585e-4dd7-ae89-4626bc1e1a06" path="/var/lib/kubelet/pods/fb260fb9-585e-4dd7-ae89-4626bc1e1a06/volumes"
Jul 09 22:49:16.626203 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:16.626162    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:16.626654 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:16.626220    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:16.638849 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:16.638817    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:17.046178 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:17.046149    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:49:18.800106 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:18.800062    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:49:47.900897 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.900866    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"]
Jul 09 22:49:47.901332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.901272    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" containerName="main"
Jul 09 22:49:47.901332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.901288    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" containerName="main"
Jul 09 22:49:47.901332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.901305    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb260fb9-585e-4dd7-ae89-4626bc1e1a06" containerName="main"
Jul 09 22:49:47.901332 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.901313    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb260fb9-585e-4dd7-ae89-4626bc1e1a06" containerName="main"
Jul 09 22:49:47.901479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.901390    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="312dd62e-a4f3-4ac8-9300-ddfcd8f659bb" containerName="main"
Jul 09 22:49:47.901479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.901403    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb260fb9-585e-4dd7-ae89-4626bc1e1a06" containerName="main"
Jul 09 22:49:47.904403 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.904387    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:47.907603 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.907584    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"openshift-service-ca.crt\""
Jul 09 22:49:47.907757 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.907714    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"kube-root-ca.crt\""
Jul 09 22:49:47.907820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.907802    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"default-dockercfg-bbv9x\""
Jul 09 22:49:47.907870 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.907810    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"sub-test-model-d3fb60-kserve-self-signed-certs\""
Jul 09 22:49:47.915527 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:47.915509    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"]
Jul 09 22:49:48.040980 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.040956    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-dshm\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.041099 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.040994    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6460e352-a659-452e-9a5b-61f11ff09d4b-tls-certs\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.041099 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.041030    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-model-cache\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.041224 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.041105    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-tmp-dir\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.041224 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.041160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4qn\" (UniqueName: \"kubernetes.io/projected/6460e352-a659-452e-9a5b-61f11ff09d4b-kube-api-access-9x4qn\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.041350 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.041226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-home\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.141987 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.141957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-tmp-dir\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142113 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142004    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4qn\" (UniqueName: \"kubernetes.io/projected/6460e352-a659-452e-9a5b-61f11ff09d4b-kube-api-access-9x4qn\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142113 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142053    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-home\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142113 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142080    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-dshm\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142316 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6460e352-a659-452e-9a5b-61f11ff09d4b-tls-certs\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142316 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-model-cache\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142472 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142449    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-tmp-dir\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142534 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142473    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-home\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.142583 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.142528    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-model-cache\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.144372 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.144351    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-dshm\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.144665 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.144647    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6460e352-a659-452e-9a5b-61f11ff09d4b-tls-certs\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.150713 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.150695    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4qn\" (UniqueName: \"kubernetes.io/projected/6460e352-a659-452e-9a5b-61f11ff09d4b-kube-api-access-9x4qn\") pod \"sub-test-model-d3fb60-kserve-b9849b5d9-79m46\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") " pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.213950 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.213925    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:48.360048 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:48.360022    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"]
Jul 09 22:49:48.362897 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:49:48.362871    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6460e352_a659_452e_9a5b_61f11ff09d4b.slice/crio-8b4e1e4236bb9137b84be6c48eaedabf4e617befff682290f0bf4ec730b4ed5d WatchSource:0}: Error finding container 8b4e1e4236bb9137b84be6c48eaedabf4e617befff682290f0bf4ec730b4ed5d: Status 404 returned error can't find the container with id 8b4e1e4236bb9137b84be6c48eaedabf4e617befff682290f0bf4ec730b4ed5d
Jul 09 22:49:49.201908 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:49.201871    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46" event={"ID":"6460e352-a659-452e-9a5b-61f11ff09d4b","Type":"ContainerStarted","Data":"466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19"}
Jul 09 22:49:49.202328 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:49.201915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46" event={"ID":"6460e352-a659-452e-9a5b-61f11ff09d4b","Type":"ContainerStarted","Data":"8b4e1e4236bb9137b84be6c48eaedabf4e617befff682290f0bf4ec730b4ed5d"}
Jul 09 22:49:49.220477 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:49.220428    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46" podStartSLOduration=2.220412821 podStartE2EDuration="2.220412821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:49:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:49:49.218916045 +0000 UTC m=+2469.109946560" watchObservedRunningTime="2026-07-09 22:49:49.220412821 +0000 UTC m=+2469.111443312"
Jul 09 22:49:52.491396 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:52.491361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:49:58.214385 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:58.214333    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:58.214965 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:58.214416    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:58.226971 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:58.226949    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:49:58.256343 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:49:58.256318    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:50:13.693450 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:13.693415    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:50:34.186892 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:34.186806    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:50:39.483147 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:39.483109    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"]
Jul 09 22:50:40.491729 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:40.491687    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"]
Jul 09 22:50:43.802591 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.802515    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"]
Jul 09 22:50:43.806099 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.806081    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:43.808444 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.808424    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a70a31\"/\"rate-test-model-a70a31-kserve-self-signed-certs\""
Jul 09 22:50:43.816582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.816556    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"]
Jul 09 22:50:43.950784 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.950745    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b65df\" (UniqueName: \"kubernetes.io/projected/bf560375-7085-4b3c-bdfa-1899359bc117-kube-api-access-b65df\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:43.950784 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.950790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-tmp-dir\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:43.951082 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.950836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-model-cache\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:43.951082 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.950922    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-dshm\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:43.951082 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.950944    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf560375-7085-4b3c-bdfa-1899359bc117-tls-certs\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:43.951082 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:43.950969    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-home\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.051557 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.051519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-model-cache\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.051767 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.051609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-dshm\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.051767 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.051632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf560375-7085-4b3c-bdfa-1899359bc117-tls-certs\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.051767 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.051653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-home\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.051767 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.051681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b65df\" (UniqueName: \"kubernetes.io/projected/bf560375-7085-4b3c-bdfa-1899359bc117-kube-api-access-b65df\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.051767 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.051700    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-tmp-dir\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.052048 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.052007    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-model-cache\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.052105 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.052075    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-tmp-dir\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.052170 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.052112    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-home\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.053871 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.053817    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-dshm\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.054180 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.054162    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf560375-7085-4b3c-bdfa-1899359bc117-tls-certs\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.062446 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.062420    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b65df\" (UniqueName: \"kubernetes.io/projected/bf560375-7085-4b3c-bdfa-1899359bc117-kube-api-access-b65df\") pod \"rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") " pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.115116 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.115064    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:44.249933 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.249908    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"]
Jul 09 22:50:44.252138 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:50:44.252106    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf560375_7085_4b3c_bdfa_1899359bc117.slice/crio-50247e144630a65e35a48aa88304d536b7da223246ef059b1a5a1dfe9a4b6def WatchSource:0}: Error finding container 50247e144630a65e35a48aa88304d536b7da223246ef059b1a5a1dfe9a4b6def: Status 404 returned error can't find the container with id 50247e144630a65e35a48aa88304d536b7da223246ef059b1a5a1dfe9a4b6def
Jul 09 22:50:44.472264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.472215    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm" event={"ID":"bf560375-7085-4b3c-bdfa-1899359bc117","Type":"ContainerStarted","Data":"c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4"}
Jul 09 22:50:44.472264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.472272    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm" event={"ID":"bf560375-7085-4b3c-bdfa-1899359bc117","Type":"ContainerStarted","Data":"50247e144630a65e35a48aa88304d536b7da223246ef059b1a5a1dfe9a4b6def"}
Jul 09 22:50:44.490001 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:44.489956    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm" podStartSLOduration=1.489942669 podStartE2EDuration="1.489942669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:50:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:50:44.488373216 +0000 UTC m=+2524.379403731" watchObservedRunningTime="2026-07-09 22:50:44.489942669 +0000 UTC m=+2524.380973183"
Jul 09 22:50:53.902047 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:53.901997    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:50:54.115516 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.115469    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:54.115705 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.115541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:54.128408 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.128379    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:54.503802 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.503754    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s" podUID="5f8239b9-94e2-4914-8173-c666a2826066" containerName="main" containerID="cri-o://d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a" gracePeriod=45
Jul 09 22:50:54.536142 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.536105    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:50:54.767203 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.767135    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:50:54.958312 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958275    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-tmp-dir\") pod \"5f8239b9-94e2-4914-8173-c666a2826066\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") "
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958331    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzwz5\" (UniqueName: \"kubernetes.io/projected/5f8239b9-94e2-4914-8173-c666a2826066-kube-api-access-gzwz5\") pod \"5f8239b9-94e2-4914-8173-c666a2826066\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") "
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958354    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-home\") pod \"5f8239b9-94e2-4914-8173-c666a2826066\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") "
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958376    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f8239b9-94e2-4914-8173-c666a2826066-tls-certs\") pod \"5f8239b9-94e2-4914-8173-c666a2826066\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") "
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958472    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-dshm\") pod \"5f8239b9-94e2-4914-8173-c666a2826066\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") "
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958496    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-model-cache\") pod \"5f8239b9-94e2-4914-8173-c666a2826066\" (UID: \"5f8239b9-94e2-4914-8173-c666a2826066\") "
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958580    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5f8239b9-94e2-4914-8173-c666a2826066" (UID: "5f8239b9-94e2-4914-8173-c666a2826066"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:54.958740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958662    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-home" (OuterVolumeSpecName: "home") pod "5f8239b9-94e2-4914-8173-c666a2826066" (UID: "5f8239b9-94e2-4914-8173-c666a2826066"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:54.959026 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958745    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-model-cache" (OuterVolumeSpecName: "model-cache") pod "5f8239b9-94e2-4914-8173-c666a2826066" (UID: "5f8239b9-94e2-4914-8173-c666a2826066"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:54.959026 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958839    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:54.959026 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958851    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:54.959026 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.958859    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:54.960764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.960725    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f8239b9-94e2-4914-8173-c666a2826066-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5f8239b9-94e2-4914-8173-c666a2826066" (UID: "5f8239b9-94e2-4914-8173-c666a2826066"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:50:54.960764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.960739    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8239b9-94e2-4914-8173-c666a2826066-kube-api-access-gzwz5" (OuterVolumeSpecName: "kube-api-access-gzwz5") pod "5f8239b9-94e2-4914-8173-c666a2826066" (UID: "5f8239b9-94e2-4914-8173-c666a2826066"). InnerVolumeSpecName "kube-api-access-gzwz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:50:54.960764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:54.960736    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-dshm" (OuterVolumeSpecName: "dshm") pod "5f8239b9-94e2-4914-8173-c666a2826066" (UID: "5f8239b9-94e2-4914-8173-c666a2826066"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:55.060330 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.060206    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzwz5\" (UniqueName: \"kubernetes.io/projected/5f8239b9-94e2-4914-8173-c666a2826066-kube-api-access-gzwz5\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:55.060330 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.060239    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f8239b9-94e2-4914-8173-c666a2826066-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:55.060330 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.060278    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f8239b9-94e2-4914-8173-c666a2826066-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:55.512618 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.512576    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46" podUID="6460e352-a659-452e-9a5b-61f11ff09d4b" containerName="main" containerID="cri-o://466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19" gracePeriod=45
Jul 09 22:50:55.528214 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.528179    2566 generic.go:356] "Generic (PLEG): container finished" podID="5f8239b9-94e2-4914-8173-c666a2826066" containerID="d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a" exitCode=0
Jul 09 22:50:55.528381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.528235    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"
Jul 09 22:50:55.528381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.528317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s" event={"ID":"5f8239b9-94e2-4914-8173-c666a2826066","Type":"ContainerDied","Data":"d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a"}
Jul 09 22:50:55.528381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.528345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s" event={"ID":"5f8239b9-94e2-4914-8173-c666a2826066","Type":"ContainerDied","Data":"e35edfe99d7654c29a2c632bc757d02a950f77a90a29c80fd03453ed87ae6fd6"}
Jul 09 22:50:55.528381 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.528362    2566 scope.go:117] "RemoveContainer" containerID="d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a"
Jul 09 22:50:55.594288 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.594259    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"]
Jul 09 22:50:55.596943 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.596920    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/sub-test-model-a70a31-kserve-64477bdbf7-g4q2s"]
Jul 09 22:50:55.623647 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.623622    2566 scope.go:117] "RemoveContainer" containerID="d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a"
Jul 09 22:50:55.624015 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:50:55.623990    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a\": container with ID starting with d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a not found: ID does not exist" containerID="d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a"
Jul 09 22:50:55.624070 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.624025    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a"} err="failed to get container status \"d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a\": rpc error: code = NotFound desc = could not find container \"d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a\": container with ID starting with d2b2f7ee6b7748e3d118c3c7887858ee82e3c35f743857eb4b5a2a0af1fcf54a not found: ID does not exist"
Jul 09 22:50:55.680198 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.680167    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8239b9-94e2-4914-8173-c666a2826066" path="/var/lib/kubelet/pods/5f8239b9-94e2-4914-8173-c666a2826066/volumes"
Jul 09 22:50:55.773339 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.773278    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:50:55.968379 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968347    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6460e352-a659-452e-9a5b-61f11ff09d4b-tls-certs\") pod \"6460e352-a659-452e-9a5b-61f11ff09d4b\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") "
Jul 09 22:50:55.968379 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968384    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-model-cache\") pod \"6460e352-a659-452e-9a5b-61f11ff09d4b\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") "
Jul 09 22:50:55.968844 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968441    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-dshm\") pod \"6460e352-a659-452e-9a5b-61f11ff09d4b\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") "
Jul 09 22:50:55.968844 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968551    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-tmp-dir\") pod \"6460e352-a659-452e-9a5b-61f11ff09d4b\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") "
Jul 09 22:50:55.968844 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968671    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x4qn\" (UniqueName: \"kubernetes.io/projected/6460e352-a659-452e-9a5b-61f11ff09d4b-kube-api-access-9x4qn\") pod \"6460e352-a659-452e-9a5b-61f11ff09d4b\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") "
Jul 09 22:50:55.968844 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968691    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6460e352-a659-452e-9a5b-61f11ff09d4b" (UID: "6460e352-a659-452e-9a5b-61f11ff09d4b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:55.968844 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968713    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-home\") pod \"6460e352-a659-452e-9a5b-61f11ff09d4b\" (UID: \"6460e352-a659-452e-9a5b-61f11ff09d4b\") "
Jul 09 22:50:55.968844 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968785    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6460e352-a659-452e-9a5b-61f11ff09d4b" (UID: "6460e352-a659-452e-9a5b-61f11ff09d4b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:55.969169 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968973    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-home" (OuterVolumeSpecName: "home") pod "6460e352-a659-452e-9a5b-61f11ff09d4b" (UID: "6460e352-a659-452e-9a5b-61f11ff09d4b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:55.969169 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.968999    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:55.969169 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.969027    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:55.970551 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.970526    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6460e352-a659-452e-9a5b-61f11ff09d4b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6460e352-a659-452e-9a5b-61f11ff09d4b" (UID: "6460e352-a659-452e-9a5b-61f11ff09d4b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:50:55.970724 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.970706    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-dshm" (OuterVolumeSpecName: "dshm") pod "6460e352-a659-452e-9a5b-61f11ff09d4b" (UID: "6460e352-a659-452e-9a5b-61f11ff09d4b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:50:55.970793 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:55.970724    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6460e352-a659-452e-9a5b-61f11ff09d4b-kube-api-access-9x4qn" (OuterVolumeSpecName: "kube-api-access-9x4qn") pod "6460e352-a659-452e-9a5b-61f11ff09d4b" (UID: "6460e352-a659-452e-9a5b-61f11ff09d4b"). InnerVolumeSpecName "kube-api-access-9x4qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:50:56.069899 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.069809    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x4qn\" (UniqueName: \"kubernetes.io/projected/6460e352-a659-452e-9a5b-61f11ff09d4b-kube-api-access-9x4qn\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:56.069899 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.069840    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:56.069899 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.069851    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6460e352-a659-452e-9a5b-61f11ff09d4b-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:56.069899 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.069860    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6460e352-a659-452e-9a5b-61f11ff09d4b-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:50:56.535590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.535554    2566 generic.go:356] "Generic (PLEG): container finished" podID="6460e352-a659-452e-9a5b-61f11ff09d4b" containerID="466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19" exitCode=0
Jul 09 22:50:56.535759 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.535595    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"
Jul 09 22:50:56.535759 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.535596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46" event={"ID":"6460e352-a659-452e-9a5b-61f11ff09d4b","Type":"ContainerDied","Data":"466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19"}
Jul 09 22:50:56.535759 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.535720    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46" event={"ID":"6460e352-a659-452e-9a5b-61f11ff09d4b","Type":"ContainerDied","Data":"8b4e1e4236bb9137b84be6c48eaedabf4e617befff682290f0bf4ec730b4ed5d"}
Jul 09 22:50:56.535759 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.535738    2566 scope.go:117] "RemoveContainer" containerID="466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19"
Jul 09 22:50:56.545510 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.545493    2566 scope.go:117] "RemoveContainer" containerID="466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19"
Jul 09 22:50:56.545740 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:50:56.545719    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19\": container with ID starting with 466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19 not found: ID does not exist" containerID="466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19"
Jul 09 22:50:56.545796 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.545753    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19"} err="failed to get container status \"466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19\": rpc error: code = NotFound desc = could not find container \"466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19\": container with ID starting with 466c41ec1a3328a43632477313b285d24973b9cf5de1f0060b70f8c935f2aa19 not found: ID does not exist"
Jul 09 22:50:56.558055 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.558027    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"]
Jul 09 22:50:56.561644 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:56.561621    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/sub-test-model-d3fb60-kserve-b9849b5d9-79m46"]
Jul 09 22:50:57.686854 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:50:57.686819    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6460e352-a659-452e-9a5b-61f11ff09d4b" path="/var/lib/kubelet/pods/6460e352-a659-452e-9a5b-61f11ff09d4b/volumes"
Jul 09 22:51:41.888023 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:41.887996    2566 scope.go:117] "RemoveContainer" containerID="96596f3ed39f1059ccc0025a846724db55f07e4b6f50401b75662130553395ba"
Jul 09 22:51:44.095479 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095447    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"]
Jul 09 22:51:44.095860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095811    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f8239b9-94e2-4914-8173-c666a2826066" containerName="main"
Jul 09 22:51:44.095860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095822    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f8239b9-94e2-4914-8173-c666a2826066" containerName="main"
Jul 09 22:51:44.095860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095844    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6460e352-a659-452e-9a5b-61f11ff09d4b" containerName="main"
Jul 09 22:51:44.095860 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095849    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6460e352-a659-452e-9a5b-61f11ff09d4b" containerName="main"
Jul 09 22:51:44.096008 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095911    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6460e352-a659-452e-9a5b-61f11ff09d4b" containerName="main"
Jul 09 22:51:44.096008 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.095924    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f8239b9-94e2-4914-8173-c666a2826066" containerName="main"
Jul 09 22:51:44.100143 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.100119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.103322 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.103297    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"kube-root-ca.crt\""
Jul 09 22:51:44.103442 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.103321    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"rate-test-model-d3fb60-kserve-self-signed-certs\""
Jul 09 22:51:44.103442 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.103328    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"default-dockercfg-bbv9x\""
Jul 09 22:51:44.103442 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.103297    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d3fb60\"/\"openshift-service-ca.crt\""
Jul 09 22:51:44.109416 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.109329    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"]
Jul 09 22:51:44.198842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.198804    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-model-cache\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.198842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.198844    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tls-certs\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.199076 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.198866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-home\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.199076 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.198950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-dshm\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.199076 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.198995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffvz9\" (UniqueName: \"kubernetes.io/projected/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-kube-api-access-ffvz9\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.199076 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.199013    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tmp-dir\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.299503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.299469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-dshm\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.299503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.299512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffvz9\" (UniqueName: \"kubernetes.io/projected/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-kube-api-access-ffvz9\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.299742 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.299533    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tmp-dir\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.299742 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.299591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-model-cache\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.299742 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.299611    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tls-certs\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.299742 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.299635    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-home\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.300050 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.300019    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-model-cache\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.300142 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.300032    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tmp-dir\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.300142 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.300068    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-home\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.301941 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.301916    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-dshm\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.302065 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.302031    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tls-certs\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.307679 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.307645    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffvz9\" (UniqueName: \"kubernetes.io/projected/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-kube-api-access-ffvz9\") pod \"rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") " pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.412872 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.412773    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:44.547489 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.547459    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"]
Jul 09 22:51:44.550863 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:51:44.550839    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e7b9ae3_3955_4d94_a9e2_22ae2535fbf3.slice/crio-7c8cf79020cdeadc5656967ea1ba909740590441c5f2dde8c83632cf6e6d5b0a WatchSource:0}: Error finding container 7c8cf79020cdeadc5656967ea1ba909740590441c5f2dde8c83632cf6e6d5b0a: Status 404 returned error can't find the container with id 7c8cf79020cdeadc5656967ea1ba909740590441c5f2dde8c83632cf6e6d5b0a
Jul 09 22:51:44.776045 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.776010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5" event={"ID":"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3","Type":"ContainerStarted","Data":"1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0"}
Jul 09 22:51:44.776217 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.776057    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5" event={"ID":"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3","Type":"ContainerStarted","Data":"7c8cf79020cdeadc5656967ea1ba909740590441c5f2dde8c83632cf6e6d5b0a"}
Jul 09 22:51:44.793674 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:44.793623    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5" podStartSLOduration=0.793607893 podStartE2EDuration="793.607893ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:51:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:51:44.792523029 +0000 UTC m=+2584.683553543" watchObservedRunningTime="2026-07-09 22:51:44.793607893 +0000 UTC m=+2584.684638407"
Jul 09 22:51:48.596217 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:48.596179    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:51:54.412915 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:54.412873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:54.413348 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:54.413015    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:54.426060 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:54.426035    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:51:54.836435 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:51:54.836403    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:52:03.597062 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:03.597012    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:52:23.386648 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:23.386611    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:52:27.509815 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:27.509773    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"]
Jul 09 22:52:28.352885 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:28.352853    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"]
Jul 09 22:52:29.300741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.300699    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7319-140482086128560"]
Jul 09 22:52:29.304233 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.304209    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:29.306897 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.306871    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-hsc52\""
Jul 09 22:52:29.313390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.313364    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128560"]
Jul 09 22:52:29.320755 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.320719    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhzn6\" (UniqueName: \"kubernetes.io/projected/263ae225-d42e-4fcc-9408-6ff73ad31b28-kube-api-access-fhzn6\") pod \"test-curl-7319-140482086128560\" (UID: \"263ae225-d42e-4fcc-9408-6ff73ad31b28\") " pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:29.421415 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.421375    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhzn6\" (UniqueName: \"kubernetes.io/projected/263ae225-d42e-4fcc-9408-6ff73ad31b28-kube-api-access-fhzn6\") pod \"test-curl-7319-140482086128560\" (UID: \"263ae225-d42e-4fcc-9408-6ff73ad31b28\") " pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:29.429946 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.429907    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhzn6\" (UniqueName: \"kubernetes.io/projected/263ae225-d42e-4fcc-9408-6ff73ad31b28-kube-api-access-fhzn6\") pod \"test-curl-7319-140482086128560\" (UID: \"263ae225-d42e-4fcc-9408-6ff73ad31b28\") " pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:29.615400 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.615294    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:29.762845 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.762818    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128560"]
Jul 09 22:52:29.764943 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:52:29.764912    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod263ae225_d42e_4fcc_9408_6ff73ad31b28.slice/crio-21381cc57106bbd8c79a6dec00813103917a96660e5ac1c70cc20ba77d6b333f WatchSource:0}: Error finding container 21381cc57106bbd8c79a6dec00813103917a96660e5ac1c70cc20ba77d6b333f: Status 404 returned error can't find the container with id 21381cc57106bbd8c79a6dec00813103917a96660e5ac1c70cc20ba77d6b333f
Jul 09 22:52:29.766217 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.766200    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 22:52:29.994187 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:29.994150    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482086128560" event={"ID":"263ae225-d42e-4fcc-9408-6ff73ad31b28","Type":"ContainerStarted","Data":"21381cc57106bbd8c79a6dec00813103917a96660e5ac1c70cc20ba77d6b333f"}
Jul 09 22:52:31.000901 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:31.000862    2566 generic.go:356] "Generic (PLEG): container finished" podID="263ae225-d42e-4fcc-9408-6ff73ad31b28" containerID="d46ac0300db570f389997560495a5c0463655c7cba4af68168e45d14593630f3" exitCode=0
Jul 09 22:52:31.001342 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:31.000942    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482086128560" event={"ID":"263ae225-d42e-4fcc-9408-6ff73ad31b28","Type":"ContainerDied","Data":"d46ac0300db570f389997560495a5c0463655c7cba4af68168e45d14593630f3"}
Jul 09 22:52:32.146651 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.146627    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:32.178921 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.178887    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7319-140482086128560_263ae225-d42e-4fcc-9408-6ff73ad31b28/test-curl-7319-140482086128560/0.log"
Jul 09 22:52:32.202575 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.202540    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128560"]
Jul 09 22:52:32.206436 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.206404    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128560"]
Jul 09 22:52:32.245387 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.245363    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhzn6\" (UniqueName: \"kubernetes.io/projected/263ae225-d42e-4fcc-9408-6ff73ad31b28-kube-api-access-fhzn6\") pod \"263ae225-d42e-4fcc-9408-6ff73ad31b28\" (UID: \"263ae225-d42e-4fcc-9408-6ff73ad31b28\") "
Jul 09 22:52:32.247619 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.247595    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263ae225-d42e-4fcc-9408-6ff73ad31b28-kube-api-access-fhzn6" (OuterVolumeSpecName: "kube-api-access-fhzn6") pod "263ae225-d42e-4fcc-9408-6ff73ad31b28" (UID: "263ae225-d42e-4fcc-9408-6ff73ad31b28"). InnerVolumeSpecName "kube-api-access-fhzn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:32.327347 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.327263    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7319-140482086128432"]
Jul 09 22:52:32.327645 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.327631    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263ae225-d42e-4fcc-9408-6ff73ad31b28" containerName="test-curl-7319-140482086128560"
Jul 09 22:52:32.327706 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.327646    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="263ae225-d42e-4fcc-9408-6ff73ad31b28" containerName="test-curl-7319-140482086128560"
Jul 09 22:52:32.327741 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.327723    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="263ae225-d42e-4fcc-9408-6ff73ad31b28" containerName="test-curl-7319-140482086128560"
Jul 09 22:52:32.330800 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.330784    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:32.334189 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.334155    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128432"]
Jul 09 22:52:32.346317 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.346290    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhzn6\" (UniqueName: \"kubernetes.io/projected/263ae225-d42e-4fcc-9408-6ff73ad31b28-kube-api-access-fhzn6\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:32.446721 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.446687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgxj\" (UniqueName: \"kubernetes.io/projected/6acf80d2-da0a-4586-aa7f-6b35e81d34ea-kube-api-access-5tgxj\") pod \"test-curl-7319-140482086128432\" (UID: \"6acf80d2-da0a-4586-aa7f-6b35e81d34ea\") " pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:32.547918 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.547881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgxj\" (UniqueName: \"kubernetes.io/projected/6acf80d2-da0a-4586-aa7f-6b35e81d34ea-kube-api-access-5tgxj\") pod \"test-curl-7319-140482086128432\" (UID: \"6acf80d2-da0a-4586-aa7f-6b35e81d34ea\") " pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:32.556038 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.556004    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgxj\" (UniqueName: \"kubernetes.io/projected/6acf80d2-da0a-4586-aa7f-6b35e81d34ea-kube-api-access-5tgxj\") pod \"test-curl-7319-140482086128432\" (UID: \"6acf80d2-da0a-4586-aa7f-6b35e81d34ea\") " pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:32.641651 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.641540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:32.774911 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:32.774884    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128432"]
Jul 09 22:52:32.777098 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:52:32.777072    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6acf80d2_da0a_4586_aa7f_6b35e81d34ea.slice/crio-add5b3011c13ab55ae44ecdf66e19440ce5246c8083bf33a86a8a34757cf91f8 WatchSource:0}: Error finding container add5b3011c13ab55ae44ecdf66e19440ce5246c8083bf33a86a8a34757cf91f8: Status 404 returned error can't find the container with id add5b3011c13ab55ae44ecdf66e19440ce5246c8083bf33a86a8a34757cf91f8
Jul 09 22:52:33.010991 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:33.010952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482086128432" event={"ID":"6acf80d2-da0a-4586-aa7f-6b35e81d34ea","Type":"ContainerStarted","Data":"add5b3011c13ab55ae44ecdf66e19440ce5246c8083bf33a86a8a34757cf91f8"}
Jul 09 22:52:33.012279 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:33.012239    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21381cc57106bbd8c79a6dec00813103917a96660e5ac1c70cc20ba77d6b333f"
Jul 09 22:52:33.012279 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:33.012268    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:33.022292 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:52:33.022237    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7319-140482086128560\" is forbidden: User \"system:node:ip-10-0-141-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-141-248.ec2.internal' and this object" podUID="263ae225-d42e-4fcc-9408-6ff73ad31b28" pod="openshift-ingress/test-curl-7319-140482086128560"
Jul 09 22:52:33.679651 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:33.679618    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="263ae225-d42e-4fcc-9408-6ff73ad31b28" path="/var/lib/kubelet/pods/263ae225-d42e-4fcc-9408-6ff73ad31b28/volumes"
Jul 09 22:52:34.018355 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:34.018319    2566 generic.go:356] "Generic (PLEG): container finished" podID="6acf80d2-da0a-4586-aa7f-6b35e81d34ea" containerID="f5d1c9b109cfdec56ad5f38ef8741fe240a8425f2710290f1f9ec78d270a0d17" exitCode=0
Jul 09 22:52:34.018556 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:34.018415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482086128432" event={"ID":"6acf80d2-da0a-4586-aa7f-6b35e81d34ea","Type":"ContainerDied","Data":"f5d1c9b109cfdec56ad5f38ef8741fe240a8425f2710290f1f9ec78d270a0d17"}
Jul 09 22:52:35.169446 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.169416    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:35.192013 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.191984    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7319-140482086128432_6acf80d2-da0a-4586-aa7f-6b35e81d34ea/test-curl-7319-140482086128432/0.log"
Jul 09 22:52:35.215613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.215576    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128432"]
Jul 09 22:52:35.218890 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.218864    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7319-140482086128432"]
Jul 09 22:52:35.273713 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.273685    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tgxj\" (UniqueName: \"kubernetes.io/projected/6acf80d2-da0a-4586-aa7f-6b35e81d34ea-kube-api-access-5tgxj\") pod \"6acf80d2-da0a-4586-aa7f-6b35e81d34ea\" (UID: \"6acf80d2-da0a-4586-aa7f-6b35e81d34ea\") "
Jul 09 22:52:35.275855 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.275792    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6acf80d2-da0a-4586-aa7f-6b35e81d34ea-kube-api-access-5tgxj" (OuterVolumeSpecName: "kube-api-access-5tgxj") pod "6acf80d2-da0a-4586-aa7f-6b35e81d34ea" (UID: "6acf80d2-da0a-4586-aa7f-6b35e81d34ea"). InnerVolumeSpecName "kube-api-access-5tgxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:35.370791 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.370757    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:35.371168 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.371155    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6acf80d2-da0a-4586-aa7f-6b35e81d34ea" containerName="test-curl-7319-140482086128432"
Jul 09 22:52:35.371214 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.371170    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6acf80d2-da0a-4586-aa7f-6b35e81d34ea" containerName="test-curl-7319-140482086128432"
Jul 09 22:52:35.371264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.371232    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6acf80d2-da0a-4586-aa7f-6b35e81d34ea" containerName="test-curl-7319-140482086128432"
Jul 09 22:52:35.374189 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.374172    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:35.374546 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.374524    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tgxj\" (UniqueName: \"kubernetes.io/projected/6acf80d2-da0a-4586-aa7f-6b35e81d34ea-kube-api-access-5tgxj\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:35.384104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.384067    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:35.475942 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.475905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgbl6\" (UniqueName: \"kubernetes.io/projected/77b47a14-afd6-42a8-84d6-a728700f412e-kube-api-access-cgbl6\") pod \"test-curl-7319-140482082027392\" (UID: \"77b47a14-afd6-42a8-84d6-a728700f412e\") " pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:35.577394 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.577301    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgbl6\" (UniqueName: \"kubernetes.io/projected/77b47a14-afd6-42a8-84d6-a728700f412e-kube-api-access-cgbl6\") pod \"test-curl-7319-140482082027392\" (UID: \"77b47a14-afd6-42a8-84d6-a728700f412e\") " pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:35.586958 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.586925    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgbl6\" (UniqueName: \"kubernetes.io/projected/77b47a14-afd6-42a8-84d6-a728700f412e-kube-api-access-cgbl6\") pod \"test-curl-7319-140482082027392\" (UID: \"77b47a14-afd6-42a8-84d6-a728700f412e\") " pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:35.680134 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.680087    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6acf80d2-da0a-4586-aa7f-6b35e81d34ea" path="/var/lib/kubelet/pods/6acf80d2-da0a-4586-aa7f-6b35e81d34ea/volumes"
Jul 09 22:52:35.684142 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.684111    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:35.829539 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:35.829510    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:35.830902 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:52:35.830873    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77b47a14_afd6_42a8_84d6_a728700f412e.slice/crio-e95189d1625f330ccc8ab69dc6ce3cf5ad74a28e443115d24f96b1291d2382c5 WatchSource:0}: Error finding container e95189d1625f330ccc8ab69dc6ce3cf5ad74a28e443115d24f96b1291d2382c5: Status 404 returned error can't find the container with id e95189d1625f330ccc8ab69dc6ce3cf5ad74a28e443115d24f96b1291d2382c5
Jul 09 22:52:36.029968 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:36.029938    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482086128432"
Jul 09 22:52:36.030156 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:36.029946    2566 scope.go:117] "RemoveContainer" containerID="f5d1c9b109cfdec56ad5f38ef8741fe240a8425f2710290f1f9ec78d270a0d17"
Jul 09 22:52:36.031337 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:36.031194    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082027392" event={"ID":"77b47a14-afd6-42a8-84d6-a728700f412e","Type":"ContainerStarted","Data":"e95189d1625f330ccc8ab69dc6ce3cf5ad74a28e443115d24f96b1291d2382c5"}
Jul 09 22:52:37.039257 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:37.039211    2566 generic.go:356] "Generic (PLEG): container finished" podID="77b47a14-afd6-42a8-84d6-a728700f412e" containerID="b481ce0bd3a15781584cdda4131343fc1d7ba2c46660a3b148c937371102a1d8" exitCode=0
Jul 09 22:52:37.039630 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:37.039301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082027392" event={"ID":"77b47a14-afd6-42a8-84d6-a728700f412e","Type":"ContainerDied","Data":"b481ce0bd3a15781584cdda4131343fc1d7ba2c46660a3b148c937371102a1d8"}
Jul 09 22:52:38.185521 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.185495    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:38.208599 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.208572    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7319-140482082027392_77b47a14-afd6-42a8-84d6-a728700f412e/test-curl-7319-140482082027392/0.log"
Jul 09 22:52:38.235334 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.235309    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:38.242333 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.242311    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:38.302485 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.302419    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgbl6\" (UniqueName: \"kubernetes.io/projected/77b47a14-afd6-42a8-84d6-a728700f412e-kube-api-access-cgbl6\") pod \"77b47a14-afd6-42a8-84d6-a728700f412e\" (UID: \"77b47a14-afd6-42a8-84d6-a728700f412e\") "
Jul 09 22:52:38.304525 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.304501    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77b47a14-afd6-42a8-84d6-a728700f412e-kube-api-access-cgbl6" (OuterVolumeSpecName: "kube-api-access-cgbl6") pod "77b47a14-afd6-42a8-84d6-a728700f412e" (UID: "77b47a14-afd6-42a8-84d6-a728700f412e"). InnerVolumeSpecName "kube-api-access-cgbl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:38.378829 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.378794    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:38.379186 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.379171    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b47a14-afd6-42a8-84d6-a728700f412e" containerName="test-curl-7319-140482082027392"
Jul 09 22:52:38.379238 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.379188    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b47a14-afd6-42a8-84d6-a728700f412e" containerName="test-curl-7319-140482082027392"
Jul 09 22:52:38.379304 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.379269    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b47a14-afd6-42a8-84d6-a728700f412e" containerName="test-curl-7319-140482082027392"
Jul 09 22:52:38.382205 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.382191    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:38.390808 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.390782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:38.403657 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.403637    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgbl6\" (UniqueName: \"kubernetes.io/projected/77b47a14-afd6-42a8-84d6-a728700f412e-kube-api-access-cgbl6\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:38.504999 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.504965    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8cbs\" (UniqueName: \"kubernetes.io/projected/ac207e9c-1b88-48bf-8a64-53c85acecf95-kube-api-access-r8cbs\") pod \"test-curl-7319-140482082027392\" (UID: \"ac207e9c-1b88-48bf-8a64-53c85acecf95\") " pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:38.606200 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.606122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8cbs\" (UniqueName: \"kubernetes.io/projected/ac207e9c-1b88-48bf-8a64-53c85acecf95-kube-api-access-r8cbs\") pod \"test-curl-7319-140482082027392\" (UID: \"ac207e9c-1b88-48bf-8a64-53c85acecf95\") " pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:38.615645 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.615621    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8cbs\" (UniqueName: \"kubernetes.io/projected/ac207e9c-1b88-48bf-8a64-53c85acecf95-kube-api-access-r8cbs\") pod \"test-curl-7319-140482082027392\" (UID: \"ac207e9c-1b88-48bf-8a64-53c85acecf95\") " pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:38.692271 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.692235    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:38.799740 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.799710    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:52:38.823147 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:38.822969    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:38.825122 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:52:38.825095    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac207e9c_1b88_48bf_8a64_53c85acecf95.slice/crio-4669943e0c14ea85b6eab8aca27c23747d76422e373ed1ece1dac30594f33f63 WatchSource:0}: Error finding container 4669943e0c14ea85b6eab8aca27c23747d76422e373ed1ece1dac30594f33f63: Status 404 returned error can't find the container with id 4669943e0c14ea85b6eab8aca27c23747d76422e373ed1ece1dac30594f33f63
Jul 09 22:52:39.051067 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:39.051040    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:39.051303 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:39.051073    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e95189d1625f330ccc8ab69dc6ce3cf5ad74a28e443115d24f96b1291d2382c5"
Jul 09 22:52:39.052303 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:39.052273    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082027392" event={"ID":"ac207e9c-1b88-48bf-8a64-53c85acecf95","Type":"ContainerStarted","Data":"4669943e0c14ea85b6eab8aca27c23747d76422e373ed1ece1dac30594f33f63"}
Jul 09 22:52:39.062828 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:39.062798    2566 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7319-140482082027392" oldPodUID="77b47a14-afd6-42a8-84d6-a728700f412e" podUID="ac207e9c-1b88-48bf-8a64-53c85acecf95"
Jul 09 22:52:39.686426 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:39.686392    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b47a14-afd6-42a8-84d6-a728700f412e" path="/var/lib/kubelet/pods/77b47a14-afd6-42a8-84d6-a728700f412e/volumes"
Jul 09 22:52:40.058533 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:40.058499    2566 generic.go:356] "Generic (PLEG): container finished" podID="ac207e9c-1b88-48bf-8a64-53c85acecf95" containerID="50116bce3d0406e8fcf629a45fc607bcf3144aedd0eb2690d0d623e88b59065f" exitCode=0
Jul 09 22:52:40.058727 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:40.058583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082027392" event={"ID":"ac207e9c-1b88-48bf-8a64-53c85acecf95","Type":"ContainerDied","Data":"50116bce3d0406e8fcf629a45fc607bcf3144aedd0eb2690d0d623e88b59065f"}
Jul 09 22:52:41.213154 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.213127    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:41.253390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.253354    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7319-140482082027392_ac207e9c-1b88-48bf-8a64-53c85acecf95/test-curl-7319-140482082027392/0.log"
Jul 09 22:52:41.283608 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.283572    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:41.290561 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.290535    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7319-140482082027392"]
Jul 09 22:52:41.333956 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.333862    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8cbs\" (UniqueName: \"kubernetes.io/projected/ac207e9c-1b88-48bf-8a64-53c85acecf95-kube-api-access-r8cbs\") pod \"ac207e9c-1b88-48bf-8a64-53c85acecf95\" (UID: \"ac207e9c-1b88-48bf-8a64-53c85acecf95\") "
Jul 09 22:52:41.336119 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.336090    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac207e9c-1b88-48bf-8a64-53c85acecf95-kube-api-access-r8cbs" (OuterVolumeSpecName: "kube-api-access-r8cbs") pod "ac207e9c-1b88-48bf-8a64-53c85acecf95" (UID: "ac207e9c-1b88-48bf-8a64-53c85acecf95"). InnerVolumeSpecName "kube-api-access-r8cbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:41.422799 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.422760    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:41.423168 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.423154    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac207e9c-1b88-48bf-8a64-53c85acecf95" containerName="test-curl-7319-140482082027392"
Jul 09 22:52:41.423215 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.423169    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac207e9c-1b88-48bf-8a64-53c85acecf95" containerName="test-curl-7319-140482082027392"
Jul 09 22:52:41.423273 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.423261    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac207e9c-1b88-48bf-8a64-53c85acecf95" containerName="test-curl-7319-140482082027392"
Jul 09 22:52:41.426510 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.426489    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:41.435091 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.435064    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8cbs\" (UniqueName: \"kubernetes.io/projected/ac207e9c-1b88-48bf-8a64-53c85acecf95-kube-api-access-r8cbs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:41.442336 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.442310    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:41.536298 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.536235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9x45\" (UniqueName: \"kubernetes.io/projected/4bcb1b06-3e28-47df-b83b-bc1234dff4cd-kube-api-access-d9x45\") pod \"test-curl-7319-140482082028400\" (UID: \"4bcb1b06-3e28-47df-b83b-bc1234dff4cd\") " pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:41.636842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.636749    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9x45\" (UniqueName: \"kubernetes.io/projected/4bcb1b06-3e28-47df-b83b-bc1234dff4cd-kube-api-access-d9x45\") pod \"test-curl-7319-140482082028400\" (UID: \"4bcb1b06-3e28-47df-b83b-bc1234dff4cd\") " pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:41.647035 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.647002    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9x45\" (UniqueName: \"kubernetes.io/projected/4bcb1b06-3e28-47df-b83b-bc1234dff4cd-kube-api-access-d9x45\") pod \"test-curl-7319-140482082028400\" (UID: \"4bcb1b06-3e28-47df-b83b-bc1234dff4cd\") " pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:41.680330 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.680302    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac207e9c-1b88-48bf-8a64-53c85acecf95" path="/var/lib/kubelet/pods/ac207e9c-1b88-48bf-8a64-53c85acecf95/volumes"
Jul 09 22:52:41.735839 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.735811    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:41.871643 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:41.871501    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:41.874272 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:52:41.874227    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bcb1b06_3e28_47df_b83b_bc1234dff4cd.slice/crio-f37ab7f71915dc09c48fe0522f8249175c69fc1b1819fc72bf759a8de0168cd2 WatchSource:0}: Error finding container f37ab7f71915dc09c48fe0522f8249175c69fc1b1819fc72bf759a8de0168cd2: Status 404 returned error can't find the container with id f37ab7f71915dc09c48fe0522f8249175c69fc1b1819fc72bf759a8de0168cd2
Jul 09 22:52:42.070070 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.070040    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082027392"
Jul 09 22:52:42.070275 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.070082    2566 scope.go:117] "RemoveContainer" containerID="50116bce3d0406e8fcf629a45fc607bcf3144aedd0eb2690d0d623e88b59065f"
Jul 09 22:52:42.071453 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.071428    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082028400" event={"ID":"4bcb1b06-3e28-47df-b83b-bc1234dff4cd","Type":"ContainerStarted","Data":"f37ab7f71915dc09c48fe0522f8249175c69fc1b1819fc72bf759a8de0168cd2"}
Jul 09 22:52:42.530454 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.530407    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm" podUID="bf560375-7085-4b3c-bdfa-1899359bc117" containerName="main" containerID="cri-o://c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4" gracePeriod=45
Jul 09 22:52:42.792820 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.792755    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:52:42.849664 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849636    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-tmp-dir\") pod \"bf560375-7085-4b3c-bdfa-1899359bc117\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") "
Jul 09 22:52:42.849834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849678    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf560375-7085-4b3c-bdfa-1899359bc117-tls-certs\") pod \"bf560375-7085-4b3c-bdfa-1899359bc117\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") "
Jul 09 22:52:42.849834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849725    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-dshm\") pod \"bf560375-7085-4b3c-bdfa-1899359bc117\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") "
Jul 09 22:52:42.849834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849783    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-model-cache\") pod \"bf560375-7085-4b3c-bdfa-1899359bc117\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") "
Jul 09 22:52:42.849834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849811    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-home\") pod \"bf560375-7085-4b3c-bdfa-1899359bc117\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") "
Jul 09 22:52:42.850024 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849869    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b65df\" (UniqueName: \"kubernetes.io/projected/bf560375-7085-4b3c-bdfa-1899359bc117-kube-api-access-b65df\") pod \"bf560375-7085-4b3c-bdfa-1899359bc117\" (UID: \"bf560375-7085-4b3c-bdfa-1899359bc117\") "
Jul 09 22:52:42.850024 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.849927    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bf560375-7085-4b3c-bdfa-1899359bc117" (UID: "bf560375-7085-4b3c-bdfa-1899359bc117"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:42.850132 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.850097    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-home" (OuterVolumeSpecName: "home") pod "bf560375-7085-4b3c-bdfa-1899359bc117" (UID: "bf560375-7085-4b3c-bdfa-1899359bc117"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:42.850132 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.850105    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-model-cache" (OuterVolumeSpecName: "model-cache") pod "bf560375-7085-4b3c-bdfa-1899359bc117" (UID: "bf560375-7085-4b3c-bdfa-1899359bc117"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:42.850274 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.850233    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:42.850322 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.850274    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:42.850322 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.850284    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:42.851858 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.851837    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-dshm" (OuterVolumeSpecName: "dshm") pod "bf560375-7085-4b3c-bdfa-1899359bc117" (UID: "bf560375-7085-4b3c-bdfa-1899359bc117"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:42.851944 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.851855    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf560375-7085-4b3c-bdfa-1899359bc117-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bf560375-7085-4b3c-bdfa-1899359bc117" (UID: "bf560375-7085-4b3c-bdfa-1899359bc117"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:52:42.851944 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.851896    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf560375-7085-4b3c-bdfa-1899359bc117-kube-api-access-b65df" (OuterVolumeSpecName: "kube-api-access-b65df") pod "bf560375-7085-4b3c-bdfa-1899359bc117" (UID: "bf560375-7085-4b3c-bdfa-1899359bc117"). InnerVolumeSpecName "kube-api-access-b65df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:42.951286 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.951224    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b65df\" (UniqueName: \"kubernetes.io/projected/bf560375-7085-4b3c-bdfa-1899359bc117-kube-api-access-b65df\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:42.951286 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.951282    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf560375-7085-4b3c-bdfa-1899359bc117-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:42.951286 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:42.951291    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf560375-7085-4b3c-bdfa-1899359bc117-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:43.085195 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.085089    2566 generic.go:356] "Generic (PLEG): container finished" podID="4bcb1b06-3e28-47df-b83b-bc1234dff4cd" containerID="fd33e00541d774ed0738a476994748b006aa5b7d6cd0c895ce2bf4e1f939fc96" exitCode=0
Jul 09 22:52:43.085195 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.085168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082028400" event={"ID":"4bcb1b06-3e28-47df-b83b-bc1234dff4cd","Type":"ContainerDied","Data":"fd33e00541d774ed0738a476994748b006aa5b7d6cd0c895ce2bf4e1f939fc96"}
Jul 09 22:52:43.086619 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.086592    2566 generic.go:356] "Generic (PLEG): container finished" podID="bf560375-7085-4b3c-bdfa-1899359bc117" containerID="c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4" exitCode=0
Jul 09 22:52:43.086752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.086646    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"
Jul 09 22:52:43.086752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.086671    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm" event={"ID":"bf560375-7085-4b3c-bdfa-1899359bc117","Type":"ContainerDied","Data":"c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4"}
Jul 09 22:52:43.086752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.086704    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm" event={"ID":"bf560375-7085-4b3c-bdfa-1899359bc117","Type":"ContainerDied","Data":"50247e144630a65e35a48aa88304d536b7da223246ef059b1a5a1dfe9a4b6def"}
Jul 09 22:52:43.086752 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.086721    2566 scope.go:117] "RemoveContainer" containerID="c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4"
Jul 09 22:52:43.095764 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.095748    2566 scope.go:117] "RemoveContainer" containerID="c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4"
Jul 09 22:52:43.095994 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:52:43.095973    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4\": container with ID starting with c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4 not found: ID does not exist" containerID="c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4"
Jul 09 22:52:43.096038 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.096002    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4"} err="failed to get container status \"c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4\": rpc error: code = NotFound desc = could not find container \"c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4\": container with ID starting with c8c5e3e2d4d4e3860ac410205ed6cf49333e2de0f8bbafc47f33fa1621f562e4 not found: ID does not exist"
Jul 09 22:52:43.120295 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.120261    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"]
Jul 09 22:52:43.124838 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.124811    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a70a31/rate-test-model-a70a31-kserve-7dbdd867c9-pbcnm"]
Jul 09 22:52:43.372432 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.372320    2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5" podUID="4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" containerName="main" containerID="cri-o://1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0" gracePeriod=45
Jul 09 22:52:43.625702 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.625644    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:52:43.680867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.680833    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf560375-7085-4b3c-bdfa-1899359bc117" path="/var/lib/kubelet/pods/bf560375-7085-4b3c-bdfa-1899359bc117/volumes"
Jul 09 22:52:43.759378 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759345    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-home\") pod \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") "
Jul 09 22:52:43.759378 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759380    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tmp-dir\") pod \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") "
Jul 09 22:52:43.759594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759406    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tls-certs\") pod \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") "
Jul 09 22:52:43.759594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759434    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-model-cache\") pod \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") "
Jul 09 22:52:43.759594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759547    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-dshm\") pod \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") "
Jul 09 22:52:43.759594 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759589    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffvz9\" (UniqueName: \"kubernetes.io/projected/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-kube-api-access-ffvz9\") pod \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\" (UID: \"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3\") "
Jul 09 22:52:43.759867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759676    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" (UID: "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:43.759867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759697    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-home" (OuterVolumeSpecName: "home") pod "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" (UID: "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:43.759867 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759686    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" (UID: "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:43.760030 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759958    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-home\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:43.760030 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759978    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tmp-dir\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:43.760030 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.759994    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-model-cache\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:43.761590 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.761564    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" (UID: "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 22:52:43.761747 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.761725    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-kube-api-access-ffvz9" (OuterVolumeSpecName: "kube-api-access-ffvz9") pod "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" (UID: "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3"). InnerVolumeSpecName "kube-api-access-ffvz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:43.761815 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.761726    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-dshm" (OuterVolumeSpecName: "dshm") pod "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" (UID: "4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 22:52:43.860919 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.860879    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-dshm\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:43.860919 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.860913    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffvz9\" (UniqueName: \"kubernetes.io/projected/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-kube-api-access-ffvz9\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:43.860919 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:43.860927    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3-tls-certs\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:44.093851 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.093812    2566 generic.go:356] "Generic (PLEG): container finished" podID="4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" containerID="1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0" exitCode=0
Jul 09 22:52:44.094037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.093862    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"
Jul 09 22:52:44.094037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.093883    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5" event={"ID":"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3","Type":"ContainerDied","Data":"1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0"}
Jul 09 22:52:44.094037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.093921    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5" event={"ID":"4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3","Type":"ContainerDied","Data":"7c8cf79020cdeadc5656967ea1ba909740590441c5f2dde8c83632cf6e6d5b0a"}
Jul 09 22:52:44.094037 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.093938    2566 scope.go:117] "RemoveContainer" containerID="1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0"
Jul 09 22:52:44.103504 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.103485    2566 scope.go:117] "RemoveContainer" containerID="1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0"
Jul 09 22:52:44.103818 ip-10-0-141-248 kubenswrapper[2566]: E0709 22:52:44.103799    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0\": container with ID starting with 1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0 not found: ID does not exist" containerID="1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0"
Jul 09 22:52:44.103875 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.103829    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0"} err="failed to get container status \"1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0\": rpc error: code = NotFound desc = could not find container \"1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0\": container with ID starting with 1eaf3a8a28211f9c0c0a6aca87faa4e8fdf0f9b6e372612b535b302be07226d0 not found: ID does not exist"
Jul 09 22:52:44.117657 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.117624    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"]
Jul 09 22:52:44.124873 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.124841    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d3fb60/rate-test-model-d3fb60-kserve-7b9b5fcb55-4vxp5"]
Jul 09 22:52:44.228075 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.228048    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:44.248666 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.248632    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7319-140482082028400_4bcb1b06-3e28-47df-b83b-bc1234dff4cd/test-curl-7319-140482082028400/0.log"
Jul 09 22:52:44.273128 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.273088    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:44.282091 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.282062    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:44.365351 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.365240    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9x45\" (UniqueName: \"kubernetes.io/projected/4bcb1b06-3e28-47df-b83b-bc1234dff4cd-kube-api-access-d9x45\") pod \"4bcb1b06-3e28-47df-b83b-bc1234dff4cd\" (UID: \"4bcb1b06-3e28-47df-b83b-bc1234dff4cd\") "
Jul 09 22:52:44.367396 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.367356    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bcb1b06-3e28-47df-b83b-bc1234dff4cd-kube-api-access-d9x45" (OuterVolumeSpecName: "kube-api-access-d9x45") pod "4bcb1b06-3e28-47df-b83b-bc1234dff4cd" (UID: "4bcb1b06-3e28-47df-b83b-bc1234dff4cd"). InnerVolumeSpecName "kube-api-access-d9x45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:44.415861 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.415824    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:44.416297 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416279    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bcb1b06-3e28-47df-b83b-bc1234dff4cd" containerName="test-curl-7319-140482082028400"
Jul 09 22:52:44.416390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416299    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bcb1b06-3e28-47df-b83b-bc1234dff4cd" containerName="test-curl-7319-140482082028400"
Jul 09 22:52:44.416390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416313    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf560375-7085-4b3c-bdfa-1899359bc117" containerName="main"
Jul 09 22:52:44.416390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416321    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf560375-7085-4b3c-bdfa-1899359bc117" containerName="main"
Jul 09 22:52:44.416390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416339    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" containerName="main"
Jul 09 22:52:44.416390 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416347    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" containerName="main"
Jul 09 22:52:44.416629 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416445    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" containerName="main"
Jul 09 22:52:44.416629 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416461    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bcb1b06-3e28-47df-b83b-bc1234dff4cd" containerName="test-curl-7319-140482082028400"
Jul 09 22:52:44.416629 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.416474    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf560375-7085-4b3c-bdfa-1899359bc117" containerName="main"
Jul 09 22:52:44.419509 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.419483    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:44.424503 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.424476    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:44.466114 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.466076    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9x45\" (UniqueName: \"kubernetes.io/projected/4bcb1b06-3e28-47df-b83b-bc1234dff4cd-kube-api-access-d9x45\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:44.566842 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.566799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzgpk\" (UniqueName: \"kubernetes.io/projected/c8640f2c-13cf-4fa0-aee7-bec248fdb353-kube-api-access-tzgpk\") pod \"test-curl-7319-140482082028400\" (UID: \"c8640f2c-13cf-4fa0-aee7-bec248fdb353\") " pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:44.668243 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.668163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzgpk\" (UniqueName: \"kubernetes.io/projected/c8640f2c-13cf-4fa0-aee7-bec248fdb353-kube-api-access-tzgpk\") pod \"test-curl-7319-140482082028400\" (UID: \"c8640f2c-13cf-4fa0-aee7-bec248fdb353\") " pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:44.676884 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.676855    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzgpk\" (UniqueName: \"kubernetes.io/projected/c8640f2c-13cf-4fa0-aee7-bec248fdb353-kube-api-access-tzgpk\") pod \"test-curl-7319-140482082028400\" (UID: \"c8640f2c-13cf-4fa0-aee7-bec248fdb353\") " pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:44.730016 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.729979    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:44.867636 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:44.867614    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:45.101376 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:45.101336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082028400" event={"ID":"c8640f2c-13cf-4fa0-aee7-bec248fdb353","Type":"ContainerStarted","Data":"766bddd8625bcb98ec4de4900f2daaf2cbedbe758d43aa9422b8e1c03a83b5bb"}
Jul 09 22:52:45.102562 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:45.102536    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:45.102562 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:45.102555    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f37ab7f71915dc09c48fe0522f8249175c69fc1b1819fc72bf759a8de0168cd2"
Jul 09 22:52:45.114822 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:45.114796    2566 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7319-140482082028400" oldPodUID="4bcb1b06-3e28-47df-b83b-bc1234dff4cd" podUID="c8640f2c-13cf-4fa0-aee7-bec248fdb353"
Jul 09 22:52:45.679918 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:45.679874    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bcb1b06-3e28-47df-b83b-bc1234dff4cd" path="/var/lib/kubelet/pods/4bcb1b06-3e28-47df-b83b-bc1234dff4cd/volumes"
Jul 09 22:52:45.680391 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:45.680357    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3" path="/var/lib/kubelet/pods/4e7b9ae3-3955-4d94-a9e2-22ae2535fbf3/volumes"
Jul 09 22:52:46.110029 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:46.109997    2566 generic.go:356] "Generic (PLEG): container finished" podID="c8640f2c-13cf-4fa0-aee7-bec248fdb353" containerID="9843e002fffcc7506ab830d5db120610408605910ff47410ddb980a7918408a0" exitCode=0
Jul 09 22:52:46.110208 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:46.110080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7319-140482082028400" event={"ID":"c8640f2c-13cf-4fa0-aee7-bec248fdb353","Type":"ContainerDied","Data":"9843e002fffcc7506ab830d5db120610408605910ff47410ddb980a7918408a0"}
Jul 09 22:52:47.249237 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.249211    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:47.277136 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.277096    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7319-140482082028400_c8640f2c-13cf-4fa0-aee7-bec248fdb353/test-curl-7319-140482082028400/0.log"
Jul 09 22:52:47.310473 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.310444    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:47.313783 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.313754    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7319-140482082028400"]
Jul 09 22:52:47.393824 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.393743    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzgpk\" (UniqueName: \"kubernetes.io/projected/c8640f2c-13cf-4fa0-aee7-bec248fdb353-kube-api-access-tzgpk\") pod \"c8640f2c-13cf-4fa0-aee7-bec248fdb353\" (UID: \"c8640f2c-13cf-4fa0-aee7-bec248fdb353\") "
Jul 09 22:52:47.395736 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.395703    2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8640f2c-13cf-4fa0-aee7-bec248fdb353-kube-api-access-tzgpk" (OuterVolumeSpecName: "kube-api-access-tzgpk") pod "c8640f2c-13cf-4fa0-aee7-bec248fdb353" (UID: "c8640f2c-13cf-4fa0-aee7-bec248fdb353"). InnerVolumeSpecName "kube-api-access-tzgpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 22:52:47.494931 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.494895    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzgpk\" (UniqueName: \"kubernetes.io/projected/c8640f2c-13cf-4fa0-aee7-bec248fdb353-kube-api-access-tzgpk\") on node \"ip-10-0-141-248.ec2.internal\" DevicePath \"\""
Jul 09 22:52:47.679979 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:47.679875    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8640f2c-13cf-4fa0-aee7-bec248fdb353" path="/var/lib/kubelet/pods/c8640f2c-13cf-4fa0-aee7-bec248fdb353/volumes"
Jul 09 22:52:48.121259 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:48.121226    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7319-140482082028400"
Jul 09 22:52:48.121472 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:48.121262    2566 scope.go:117] "RemoveContainer" containerID="9843e002fffcc7506ab830d5db120610408605910ff47410ddb980a7918408a0"
Jul 09 22:52:52.390634 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:52:52.390597    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:53:02.305700 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:02.305661    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c976bc6bc-v8m6q_10105218-f51a-4c6a-a6e9-2f1e000ecda4/authorino/0.log"
Jul 09 22:53:04.398349 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:04.398309    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-lnjkn"]
Jul 09 22:53:07.750462 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:07.750434    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-fpkzd_98aa773d-8adf-4867-8614-86ed24c814fb/copy-manifests/0.log"
Jul 09 22:53:07.758418 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:07.758392    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-fpkzd_98aa773d-8adf-4867-8614-86ed24c814fb/manager/0.log"
Jul 09 22:53:09.813828 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:09.813792    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c976bc6bc-v8m6q_10105218-f51a-4c6a-a6e9-2f1e000ecda4/authorino/0.log"
Jul 09 22:53:10.220435 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:10.220405    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-r2dxt_029c032c-df11-44ac-815e-59be560a8f31/kuadrant-console-plugin/0.log"
Jul 09 22:53:10.593296 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:10.593207    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-lnjkn_000ca2f2-1f10-4662-a485-329ff5a23539/limitador/0.log"
Jul 09 22:53:11.298054 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:11.298023    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-g2kn8_adfeb335-b804-4bc3-92e0-2ca883fa0534/kube-auth-proxy/0.log"
Jul 09 22:53:16.538328 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.538285    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t249s/must-gather-knblg"]
Jul 09 22:53:16.538715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.538669    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8640f2c-13cf-4fa0-aee7-bec248fdb353" containerName="test-curl-7319-140482082028400"
Jul 09 22:53:16.538715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.538680    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8640f2c-13cf-4fa0-aee7-bec248fdb353" containerName="test-curl-7319-140482082028400"
Jul 09 22:53:16.538787 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.538745    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8640f2c-13cf-4fa0-aee7-bec248fdb353" containerName="test-curl-7319-140482082028400"
Jul 09 22:53:16.543483 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.543466    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.552263 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.552228    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t249s\"/\"kube-root-ca.crt\""
Jul 09 22:53:16.552398 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.552240    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t249s\"/\"default-dockercfg-5q4rk\""
Jul 09 22:53:16.552398 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.552312    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t249s\"/\"openshift-service-ca.crt\""
Jul 09 22:53:16.586437 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.586407    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t249s/must-gather-knblg"]
Jul 09 22:53:16.661186 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.661149    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/72313d0b-16eb-4dac-bbd4-7725d6babc93-must-gather-output\") pod \"must-gather-knblg\" (UID: \"72313d0b-16eb-4dac-bbd4-7725d6babc93\") " pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.661186 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.661192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s47h\" (UniqueName: \"kubernetes.io/projected/72313d0b-16eb-4dac-bbd4-7725d6babc93-kube-api-access-9s47h\") pod \"must-gather-knblg\" (UID: \"72313d0b-16eb-4dac-bbd4-7725d6babc93\") " pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.762305 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.762230    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/72313d0b-16eb-4dac-bbd4-7725d6babc93-must-gather-output\") pod \"must-gather-knblg\" (UID: \"72313d0b-16eb-4dac-bbd4-7725d6babc93\") " pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.762305 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.762310    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s47h\" (UniqueName: \"kubernetes.io/projected/72313d0b-16eb-4dac-bbd4-7725d6babc93-kube-api-access-9s47h\") pod \"must-gather-knblg\" (UID: \"72313d0b-16eb-4dac-bbd4-7725d6babc93\") " pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.762642 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.762618    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/72313d0b-16eb-4dac-bbd4-7725d6babc93-must-gather-output\") pod \"must-gather-knblg\" (UID: \"72313d0b-16eb-4dac-bbd4-7725d6babc93\") " pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.780489 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.780454    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s47h\" (UniqueName: \"kubernetes.io/projected/72313d0b-16eb-4dac-bbd4-7725d6babc93-kube-api-access-9s47h\") pod \"must-gather-knblg\" (UID: \"72313d0b-16eb-4dac-bbd4-7725d6babc93\") " pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.853623 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.853520    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t249s/must-gather-knblg"
Jul 09 22:53:16.994295 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:16.994271    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t249s/must-gather-knblg"]
Jul 09 22:53:16.996357 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:53:16.996321    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72313d0b_16eb_4dac_bbd4_7725d6babc93.slice/crio-376e2785e0220c46028e5d48a410ad5852c2f0520c17c173ee8487117e8c43f5 WatchSource:0}: Error finding container 376e2785e0220c46028e5d48a410ad5852c2f0520c17c173ee8487117e8c43f5: Status 404 returned error can't find the container with id 376e2785e0220c46028e5d48a410ad5852c2f0520c17c173ee8487117e8c43f5
Jul 09 22:53:17.280712 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:17.280678    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/must-gather-knblg" event={"ID":"72313d0b-16eb-4dac-bbd4-7725d6babc93","Type":"ContainerStarted","Data":"376e2785e0220c46028e5d48a410ad5852c2f0520c17c173ee8487117e8c43f5"}
Jul 09 22:53:18.287407 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:18.287367    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/must-gather-knblg" event={"ID":"72313d0b-16eb-4dac-bbd4-7725d6babc93","Type":"ContainerStarted","Data":"5713cb0abf9ac563538eb845efbfe509be2096e564843edc2bbe28f43df13fad"}
Jul 09 22:53:18.287830 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:18.287419    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/must-gather-knblg" event={"ID":"72313d0b-16eb-4dac-bbd4-7725d6babc93","Type":"ContainerStarted","Data":"9c8d12a4ae4113de041cdf223102bbb39c2453ba9a794eae1f0cb86098a37844"}
Jul 09 22:53:18.306824 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:18.306760    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t249s/must-gather-knblg" podStartSLOduration=1.523549531 podStartE2EDuration="2.306737682s" totalImagesPullingTime="783.188151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 22:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:53:18.304445683 +0000 UTC m=+2678.195476198" watchObservedRunningTime="2026-07-09 22:53:18.306737682 +0000 UTC m=+2678.197768200"
Jul 09 22:53:19.547988 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:19.547948    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dx7b8_399349ba-0655-4074-b684-4e43c64ddfe3/global-pull-secret-syncer/0.log"
Jul 09 22:53:19.704104 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:19.704071    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-r76ff_59036915-3574-439b-bdd6-11bbde0bc1dc/konnectivity-agent/0.log"
Jul 09 22:53:19.825308 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:19.825216    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-248.ec2.internal_75c20c336a43f381c5b679d63baebd8c/haproxy/0.log"
Jul 09 22:53:25.373272 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:25.372011    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7c976bc6bc-v8m6q_10105218-f51a-4c6a-a6e9-2f1e000ecda4/authorino/0.log"
Jul 09 22:53:25.479666 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:25.479633    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-r2dxt_029c032c-df11-44ac-815e-59be560a8f31/kuadrant-console-plugin/0.log"
Jul 09 22:53:25.712956 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:25.712922    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-lnjkn_000ca2f2-1f10-4662-a485-329ff5a23539/limitador/0.log"
Jul 09 22:53:27.543838 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.543805    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/alertmanager/0.log"
Jul 09 22:53:27.639396 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.639365    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/config-reloader/0.log"
Jul 09 22:53:27.701330 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.701294    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/kube-rbac-proxy-web/0.log"
Jul 09 22:53:27.745126 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.745086    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/kube-rbac-proxy/0.log"
Jul 09 22:53:27.771987 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.771946    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/kube-rbac-proxy-metric/0.log"
Jul 09 22:53:27.799582 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.799483    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/prom-label-proxy/0.log"
Jul 09 22:53:27.825096 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.825064    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38e5ddaa-fc2d-4fb9-8250-34a822b0f73f/init-config-reloader/0.log"
Jul 09 22:53:27.880552 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:27.880497    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-ngfns_35412c7d-cb46-4257-857e-cdd8156e34aa/cluster-monitoring-operator/0.log"
Jul 09 22:53:28.026762 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.026672    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-2kv58_72e32bdf-a214-4b93-bcb3-5f826999bc8b/monitoring-plugin/0.log"
Jul 09 22:53:28.063925 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.063838    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dxtjf_b40b51cc-d1be-4ec8-836a-d3f9ea69c13d/node-exporter/0.log"
Jul 09 22:53:28.095789 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.095756    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dxtjf_b40b51cc-d1be-4ec8-836a-d3f9ea69c13d/kube-rbac-proxy/0.log"
Jul 09 22:53:28.121774 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.121688    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dxtjf_b40b51cc-d1be-4ec8-836a-d3f9ea69c13d/init-textfile/0.log"
Jul 09 22:53:28.342416 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.342335    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-dmfn2_6b1d3909-2181-4287-9ad6-8e5f5949bfce/kube-rbac-proxy-main/0.log"
Jul 09 22:53:28.368487 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.368453    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-dmfn2_6b1d3909-2181-4287-9ad6-8e5f5949bfce/kube-rbac-proxy-self/0.log"
Jul 09 22:53:28.396559 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.396526    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-dmfn2_6b1d3909-2181-4287-9ad6-8e5f5949bfce/openshift-state-metrics/0.log"
Jul 09 22:53:28.645872 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.645787    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7r287_042a8dd6-030e-4c41-bda8-28c7e7487284/prometheus-operator/0.log"
Jul 09 22:53:28.670194 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.670144    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-7r287_042a8dd6-030e-4c41-bda8-28c7e7487284/kube-rbac-proxy/0.log"
Jul 09 22:53:28.699264 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.699197    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-5kfjk_78ca84a6-3168-4db1-8425-574a9f7752fe/prometheus-operator-admission-webhook/0.log"
Jul 09 22:53:28.742570 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.742534    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b889f67-whbfj_3dcae647-902f-41eb-854d-f49604d5ccde/telemeter-client/0.log"
Jul 09 22:53:28.768799 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.768771    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b889f67-whbfj_3dcae647-902f-41eb-854d-f49604d5ccde/reload/0.log"
Jul 09 22:53:28.798395 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:28.798366    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69b889f67-whbfj_3dcae647-902f-41eb-854d-f49604d5ccde/kube-rbac-proxy/0.log"
Jul 09 22:53:30.107667 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.107634    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xx6fx_cd47921f-23a4-46c0-8887-ce189ee42315/networking-console-plugin/0.log"
Jul 09 22:53:30.259558 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.259523    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"]
Jul 09 22:53:30.264937 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.264914    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.271210 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.271184    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"]
Jul 09 22:53:30.408072 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.407985    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-sys\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.408337 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.408314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqhg8\" (UniqueName: \"kubernetes.io/projected/834a71c0-d09f-47ca-888f-0e2cec305615-kube-api-access-jqhg8\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.408482 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.408464    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-proc\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.408638 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.408623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-lib-modules\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.408754 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.408738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-podres\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510310 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-sys\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510597 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqhg8\" (UniqueName: \"kubernetes.io/projected/834a71c0-d09f-47ca-888f-0e2cec305615-kube-api-access-jqhg8\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510625    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-proc\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510570    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-sys\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510715 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510695    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-lib-modules\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510713    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-proc\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510834 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510721    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-podres\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510903 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510829    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-podres\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.510903 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.510863    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/834a71c0-d09f-47ca-888f-0e2cec305615-lib-modules\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.521712 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.521681    2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqhg8\" (UniqueName: \"kubernetes.io/projected/834a71c0-d09f-47ca-888f-0e2cec305615-kube-api-access-jqhg8\") pod \"perf-node-gather-daemonset-f8l7v\" (UID: \"834a71c0-d09f-47ca-888f-0e2cec305615\") " pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.577717 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.577681    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:30.763028 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:30.762900    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"]
Jul 09 22:53:30.767972 ip-10-0-141-248 kubenswrapper[2566]: W0709 22:53:30.767926    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod834a71c0_d09f_47ca_888f_0e2cec305615.slice/crio-9280d24fdd974cd85fca0d9a51b3dd2e17eb69c777539fc6979203c76990405c WatchSource:0}: Error finding container 9280d24fdd974cd85fca0d9a51b3dd2e17eb69c777539fc6979203c76990405c: Status 404 returned error can't find the container with id 9280d24fdd974cd85fca0d9a51b3dd2e17eb69c777539fc6979203c76990405c
Jul 09 22:53:31.350241 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:31.350202    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-585cb75754-z5nvn_6cc3cb92-0c1b-4d10-82be-ec18d318396d/console/0.log"
Jul 09 22:53:31.386071 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:31.386027    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v" event={"ID":"834a71c0-d09f-47ca-888f-0e2cec305615","Type":"ContainerStarted","Data":"38ba9ef4ebb51ccd633f7d69b558ac7d98b8fac499780860577f96661055feca"}
Jul 09 22:53:31.386306 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:31.386146    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v" event={"ID":"834a71c0-d09f-47ca-888f-0e2cec305615","Type":"ContainerStarted","Data":"9280d24fdd974cd85fca0d9a51b3dd2e17eb69c777539fc6979203c76990405c"}
Jul 09 22:53:31.386389 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:31.386348    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:31.410198 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:31.410142    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v" podStartSLOduration=1.410124382 podStartE2EDuration="1.410124382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 22:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 22:53:31.407311967 +0000 UTC m=+2691.298342477" watchObservedRunningTime="2026-07-09 22:53:31.410124382 +0000 UTC m=+2691.301154895"
Jul 09 22:53:32.023549 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:32.023474    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-ljxg8_5bef2150-f55b-4757-b75e-8c27464c6a5e/volume-data-source-validator/0.log"
Jul 09 22:53:33.294600 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:33.294554    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g9qfh_c5478bd7-3b16-49c6-a902-e016f8f4e82d/dns/0.log"
Jul 09 22:53:33.322158 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:33.322127    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g9qfh_c5478bd7-3b16-49c6-a902-e016f8f4e82d/kube-rbac-proxy/0.log"
Jul 09 22:53:33.351646 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:33.351618    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2r5v5_87ff8432-417c-4f78-b181-4f44077cf60f/dns-node-resolver/0.log"
Jul 09 22:53:34.044566 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:34.044513    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8z5xq_94f487f9-70be-4b90-a26c-4f235797dd0f/node-ca/0.log"
Jul 09 22:53:35.392661 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:35.392631    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-g2kn8_adfeb335-b804-4bc3-92e0-2ca883fa0534/kube-auth-proxy/0.log"
Jul 09 22:53:36.460145 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:36.460111    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2xl48_e4f8c700-e1c0-4abc-bda7-cd7154c6989d/serve-healthcheck-canary/0.log"
Jul 09 22:53:37.049773 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:37.049735    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-mbc8s_ecd5587b-22e2-4d24-a201-e37a074f83ad/insights-operator/0.log"
Jul 09 22:53:37.050570 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:37.050539    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-mbc8s_ecd5587b-22e2-4d24-a201-e37a074f83ad/insights-operator/1.log"
Jul 09 22:53:37.077640 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:37.077603    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-br8qn_405b2834-9696-49e6-94e9-97c5c2e75963/kube-rbac-proxy/0.log"
Jul 09 22:53:37.105800 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:37.105764    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-br8qn_405b2834-9696-49e6-94e9-97c5c2e75963/exporter/0.log"
Jul 09 22:53:37.135613 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:37.135576    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-br8qn_405b2834-9696-49e6-94e9-97c5c2e75963/extractor/0.log"
Jul 09 22:53:37.407855 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:37.407771    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t249s/perf-node-gather-daemonset-f8l7v"
Jul 09 22:53:40.251576 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:40.251543    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-fpkzd_98aa773d-8adf-4867-8614-86ed24c814fb/manager/0.log"
Jul 09 22:53:40.279042 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:40.279008    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5b79867d65-fpkzd_98aa773d-8adf-4867-8614-86ed24c814fb/copy-manifests/0.log"
Jul 09 22:53:41.313721 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:41.313685    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-g2kn8_adfeb335-b804-4bc3-92e0-2ca883fa0534/kube-auth-proxy/0.log"
Jul 09 22:53:41.744546 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:41.744485    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55bc84657b-47kcx_37d0979b-1ccb-45f9-9eb2-23e417511e74/manager/0.log"
Jul 09 22:53:41.812046 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:41.812013    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-m9sj6_55914045-8562-4de3-aa9e-e6a7590736df/openshift-lws-operator/0.log"
Jul 09 22:53:42.943367 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:42.943329    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7ff5d987cc-g2kn8_adfeb335-b804-4bc3-92e0-2ca883fa0534/kube-auth-proxy/0.log"
Jul 09 22:53:47.158176 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:47.158141    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-c42ff_7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc/migrator/0.log"
Jul 09 22:53:47.191065 ip-10-0-141-248 kubenswrapper[2566]: I0709 22:53:47.191036    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-c42ff_7d41b6bc-4f9d-4aa7-9c2b-30ddaa960bfc/graceful-termination/0.log"