Jul 19 14:46:38.159655 ip-10-0-135-98 systemd[1]: Starting Kubernetes Kubelet...
Jul 19 14:46:38.607935 ip-10-0-135-98 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:46:38.607935 ip-10-0-135-98 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 19 14:46:38.607935 ip-10-0-135-98 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:46:38.607935 ip-10-0-135-98 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 19 14:46:38.607935 ip-10-0-135-98 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 19 14:46:38.610045 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.609962    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 19 14:46:38.615423 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615409    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:38.615423 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615424    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615429    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615434    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615437    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615441    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615445    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615447    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615456    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615459    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615461    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615464    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615467    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615469    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615472    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615474    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615477    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615479    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615482    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615485    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615487    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:38.615485 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615490    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615493    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615496    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615498    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615501    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615504    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615506    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615508    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615512    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615514    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615516    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615519    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615521    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615524    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615526    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615529    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615531    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615533    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615536    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615538    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:38.615956 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615541    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615543    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615546    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615548    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615551    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615553    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615555    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615558    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615560    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615562    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615565    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615567    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615569    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615571    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615574    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615577    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615579    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615582    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615584    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:38.616489 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615587    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615589    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615591    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615594    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615596    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615598    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615601    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615603    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615605    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615609    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615612    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615615    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615618    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615621    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615624    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615626    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615629    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615631    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615633    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:38.616935 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615635    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615638    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615640    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615643    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615645    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615649    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615651    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615654    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615657    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615660    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615662    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615665    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615667    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615672    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615675    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615679    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615681    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615685    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615688    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:38.617404 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.615691    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616388    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616394    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616397    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616400    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616403    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616406    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616409    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616412    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616415    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616418    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616420    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616423    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616425    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616428    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616431    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616433    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616435    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616438    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616441    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:38.617848 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616445    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616447    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616450    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616452    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616455    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616457    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616460    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616463    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616466    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616468    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616470    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616473    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616475    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616478    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616480    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616482    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616485    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616487    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616490    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:38.618346 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616509    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616513    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616516    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616521    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616525    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616528    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616530    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616533    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616535    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616538    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616540    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616543    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616546    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616549    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616551    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616554    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616556    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616560    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616563    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616565    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:38.618901 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616568    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616570    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616573    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616575    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616578    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616580    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616583    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616585    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616587    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616590    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616592    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616596    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616599    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616602    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616604    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616608    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616611    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616614    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616617    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:38.619408 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616620    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616622    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616624    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616627    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616629    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616633    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616636    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616638    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616641    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616643    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616646    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616648    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616650    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616653    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616655    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616658    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616661    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616663    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616667    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:38.619857 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616670    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616673    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.616675    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617060    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617073    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617077    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617080    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617083    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617085    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617089    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617091    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617094    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617096    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617099    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617103    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617107    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617110    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617113    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617116    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:38.620358 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617118    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617121    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617124    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617126    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617128    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617131    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617134    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617136    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617139    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617141    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617144    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617146    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617148    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617151    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617153    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617156    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617158    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617163    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617166    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:38.620831 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617168    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617171    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617173    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617176    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617178    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617180    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617183    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617187    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617189    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617192    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617194    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617197    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617199    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617202    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617205    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617208    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617210    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617212    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617215    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617218    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:38.621286 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617220    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617223    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617225    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617227    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617230    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617232    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617234    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617237    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617239    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617242    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617245    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617248    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617252    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617255    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617258    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617261    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617264    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617267    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617269    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:38.621806 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617272    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617275    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617277    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617279    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617284    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617287    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617290    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617292    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617295    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617297    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617300    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617302    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617304    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617307    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617310    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617314    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617318    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617321    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617323    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617326    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:38.622271 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617329    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617342    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617345    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617348    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.617351    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617417    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617425    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617432    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617437    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617441    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617444    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617448    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617453    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617455    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617458    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617462    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617465    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617468    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617471    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617474    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617477    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617480    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617483    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617487    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 19 14:46:38.622774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617490    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617493    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617496    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617499    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617502    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617505    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617508    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617511    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617514    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617517    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617521    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617527    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617530    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617535    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617538    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617541    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617544    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617547    2572 flags.go:64] FLAG: --enable-server="true"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617549    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617554    2572 flags.go:64] FLAG: --event-burst="100"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617557    2572 flags.go:64] FLAG: --event-qps="50"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617560    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617563    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617566    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 19 14:46:38.623391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617569    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617572    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617575    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617578    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617581    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617584    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617587    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617590    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617593    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617596    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617599    2572 flags.go:64] FLAG: --feature-gates=""
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617602    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617605    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617608    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617611    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617617    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617620    2572 flags.go:64] FLAG: --help="false"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617623    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617626    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617630    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617633    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617636    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617639    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617643    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 19 14:46:38.623948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617646    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617648    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617651    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617654    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617657    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617660    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617662    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617665    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617668    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617671    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617673    2572 flags.go:64] FLAG: --lock-file=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617676    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617679    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617682    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617691    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617694    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617696    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617699    2572 flags.go:64] FLAG: --logging-format="text"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617702    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617705    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617708    2572 flags.go:64] FLAG: --manifest-url=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617711    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617715    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617719    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617723    2572 flags.go:64] FLAG: --max-pods="110"
Jul 19 14:46:38.624572 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617726    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617729    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617733    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617736    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617739    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617742    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617744    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617752    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617755    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617758    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617761    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617763    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617768    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617771    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617774    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617777    2572 flags.go:64] FLAG: --port="10250"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617780    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617782    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03c7688ac97aae57e"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617785    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617788    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617791    2572 flags.go:64] FLAG: --register-node="true"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617794    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617797    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617800    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 19 14:46:38.625163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617803    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617806    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617809    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617812    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617815    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617818    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617820    2572 flags.go:64] FLAG: --runonce="false"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617825    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617827    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617830    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617834    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617837    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617840    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617843    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617846    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617849    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617852    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617855    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617858    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617861    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617864    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617867    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617873    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617875    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617878    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 19 14:46:38.625813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617882    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617884    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617887    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617890    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617893    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617895    2572 flags.go:64] FLAG: --v="2"
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617899    2572 flags.go:64] FLAG: --version="false"
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617903    2572 flags.go:64] FLAG: --vmodule=""
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617906    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.617909    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618029    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618033    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618036    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618039    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618042    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618045    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618047    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618052    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618054    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618056    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618059    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618061    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:38.626476 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618064    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618067    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618069    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618072    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618074    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618077    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618079    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618082    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618084    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618087    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618089    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618091    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618094    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618096    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618099    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618101    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618103    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618106    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618108    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618111    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:38.627226 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618113    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618116    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618118    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618121    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618124    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618126    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618128    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618132    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618135    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618137    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618140    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618142    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618145    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618147    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618150    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618153    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618155    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618157    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618160    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:38.627907 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618162    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618166    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618170    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618172    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618176    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618178    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618181    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618186    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618188    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618191    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618194    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618197    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618199    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618202    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618204    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618207    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618209    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618212    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618214    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:38.628387 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618216    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618220    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618222    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618243    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618248    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618251    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618253    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618256    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618258    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618261    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618264    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618267    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618270    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618274    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618276    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618279    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618281    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618284    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618286    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:38.628843 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618289    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618291    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618294    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618296    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618298    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618301    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618303    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618306    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618308    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.618310    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.619049    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.628878    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 19 14:46:38.629297 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.628895    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 19 14:46:38.629776 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.629765    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 19 14:46:38.629809 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.629777    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 19 14:46:38.629844 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629835    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:38.629844 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629844    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629848    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629851    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629854    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629857    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629859    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629862    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629866    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629870    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629873    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629877    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629879    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629882    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629885    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629888    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629891    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629893    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629896    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629899    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:38.629895 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629901    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629904    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629907    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629910    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629912    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629915    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629918    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629921    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629924    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629926    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629929    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629931    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629934    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629936    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629939    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629941    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629944    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629946    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629949    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629951    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:38.630347 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629954    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629956    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629959    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629961    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629963    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629966    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629969    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629971    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629974    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629976    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629979    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629981    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629983    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629986    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629989    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629992    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629994    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629997    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.629999    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630002    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:38.630849 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630004    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630008    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630012    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630015    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630018    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630021    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630024    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630027    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630029    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630032    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630035    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630038    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630040    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630043    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630046    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630048    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630051    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630053    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630057    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630060    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:38.631320 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630062    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630065    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630067    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630070    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630072    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630075    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630077    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630080    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630082    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630085    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630088    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630091    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630094    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630097    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630100    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630103    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630106    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630108    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:38.631822 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630111    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.630116    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630223    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630228    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630231    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630233    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630237    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630240    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630243    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630245    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630248    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630251    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630253    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630256    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630259    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 19 14:46:38.632246 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630262    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630265    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630269    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630272    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630275    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630278    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630280    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630284    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630286    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630289    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630291    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630294    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630297    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630299    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630302    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630304    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630306    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630309    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630311    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 19 14:46:38.632628 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630314    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630316    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630318    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630321    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630323    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630326    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630328    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630346    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630351    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630354    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630357    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630360    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630363    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630367    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630369    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630372    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630374    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630377    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630379    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630382    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 19 14:46:38.633078 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630384    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630387    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630390    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630392    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630394    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630397    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630399    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630403    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630406    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630408    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630410    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630413    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630415    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630417    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630420    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630422    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630425    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630427    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630429    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630432    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 19 14:46:38.633631 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630434    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630436    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630439    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630441    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630444    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630446    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630450    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630453    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630456    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630459    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630462    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630465    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630468    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630471    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630474    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630476    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630479    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630481    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630484    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 19 14:46:38.634105 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630487    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630489    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630492    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630494    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630497    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630499    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630502    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:38.630504    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.630509    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 19 14:46:38.634570 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.630601    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 19 14:46:38.635400 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.635387    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 19 14:46:38.637001 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.636990    2572 server.go:1019] "Starting client certificate rotation"
Jul 19 14:46:38.637113 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.637097    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:46:38.637144 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.637135    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 19 14:46:38.640005 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.639992    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:46:38.641103 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.641081    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 19 14:46:38.654482 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.654466    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 19 14:46:38.657065 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.657047    2572 log.go:25] "Validated CRI v1 image API"
Jul 19 14:46:38.657944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.657927    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 19 14:46:38.660464 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.660447    2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d9a1bbda-1e2c-4fde-8625-07c5db0c10d4:/dev/nvme0n1p3 e3661655-b019-4699-a97a-52bea5591ff9:/dev/nvme0n1p4]
Jul 19 14:46:38.660529 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.660464    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 19 14:46:38.668941 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.668818    2572 manager.go:216] Machine: {Timestamp:2026-07-19 14:46:38.667294525 +0000 UTC m=+0.368742560 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100043 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28a13c9173c507967006aa6a571d5a SystemUUID:ec28a13c-9173-c507-9670-06aa6a571d5a BootID:2fe980e4-fa00-4808-b770-85dfd1002c51 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:bf:0b:0a:c5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:bf:0b:0a:c5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:35:94:b3:86:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 19 14:46:38.668941 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.668932    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 19 14:46:38.669079 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.669039    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 19 14:46:38.671856 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.671832    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 19 14:46:38.671983 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.671859    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-98.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 19 14:46:38.672412 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.672395    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:46:38.672683 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.672672    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 19 14:46:38.672716 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.672686    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 19 14:46:38.672716 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.672694    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:46:38.673510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.673499    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 19 14:46:38.675308 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.675297    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 19 14:46:38.678596 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678586    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:46:38.678700 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678691    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 19 14:46:38.678753 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678726    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 19 14:46:38.678753 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678736    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 19 14:46:38.678753 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678749    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 19 14:46:38.678874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678758    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 19 14:46:38.678874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.678768    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 19 14:46:38.679081 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.679069    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:46:38.679112 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.679090    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 19 14:46:38.681740 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.681724    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 19 14:46:38.684033 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.684019    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 19 14:46:38.684097 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.684049    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 19 14:46:38.685485 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685473    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685489    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685495    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685501    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685511    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685516    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685527    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685534    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 19 14:46:38.685539 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685541    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 19 14:46:38.685769 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685547    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 19 14:46:38.685769 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685556    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 19 14:46:38.685769 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.685561    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 19 14:46:38.685984 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.685965    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 19 14:46:38.686017 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.685966    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 19 14:46:38.686444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.686431    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 19 14:46:38.686444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.686442    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 19 14:46:38.686998 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.686981    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z92vf"
Jul 19 14:46:38.689444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.689431    2572 server.go:1268] "Started kubelet"
Jul 19 14:46:38.689555 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.689520    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 19 14:46:38.689619 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.689538    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 19 14:46:38.689734 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.689632    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 19 14:46:38.690190 ip-10-0-135-98 systemd[1]: Started Kubernetes Kubelet.
Jul 19 14:46:38.691502 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.691490    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 19 14:46:38.692005 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.691990    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 19 14:46:38.692657 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.692632    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-98.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 19 14:46:38.695171 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.695152    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z92vf"
Jul 19 14:46:38.695403 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.695379    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 19 14:46:38.700766 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.699592    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-98.ec2.internal.18c3b8312b5f0361  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-98.ec2.internal,UID:ip-10-0-135-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-98.ec2.internal,},FirstTimestamp:2026-07-19 14:46:38.689403745 +0000 UTC m=+0.390851790,LastTimestamp:2026-07-19 14:46:38.689403745 +0000 UTC m=+0.390851790,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-98.ec2.internal,}"
Jul 19 14:46:38.702323 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.701781    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 19 14:46:38.702323 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.701805    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 19 14:46:38.702323 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.701939    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.702651    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.702670    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.702691    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.702786    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.702793    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703009    2572 factory.go:55] Registering systemd factory
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703021    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703203    2572 factory.go:153] Registering CRI-O factory
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703213    2572 factory.go:223] Registration of the crio container factory successfully
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703251    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703268    2572 factory.go:103] Registering Raw factory
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703278    2572 manager.go:1196] Started watching for new ooms in manager
Jul 19 14:46:38.704238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.703627    2572 manager.go:349] Starting recovery of all containers
Jul 19 14:46:38.704856 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.704487    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:38.708373 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.707373    2572 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-98.ec2.internal\" not found" node="ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.714638 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.714621    2572 manager.go:354] Recovery completed
Jul 19 14:46:38.718646 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.718582    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.722659 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.722640    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.722727 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.722679    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.722727 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.722695    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.723258 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723246    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 19 14:46:38.723313 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723257    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.723313 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723257    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 19 14:46:38.723427 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723320    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 19 14:46:38.723950 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723917    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.723950 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723949    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.724037 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.723962    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.724349 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.724324    2572 policy_none.go:49] "None policy: Start"
Jul 19 14:46:38.724393 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.724354    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 19 14:46:38.724393 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.724365    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 19 14:46:38.725327 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.725318    2572 policy_none.go:47] "Start"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.769497    2572 manager.go:337] "Starting Device Plugin manager"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.769639    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.769649    2572 server.go:87] "Starting device plugin registration server"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.769862    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.769892    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.769957    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.770109    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.770124    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.770699    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.770748    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.778035    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.779306    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.779361    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.779389    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.779427    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 19 14:46:38.797264 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.781618    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:38.870634 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.870541    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.872091 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.872070    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.872166 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.872103    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.872166 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.872128    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.872232 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.872172    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.880182 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.880166    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.880240 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.880190    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-98.ec2.internal\": node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:38.880282 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.880265    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"]
Jul 19 14:46:38.880330 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.880320    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.881722 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.881696    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.881812 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.881730    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.881812 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.881745    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.884062 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884049    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.884228 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884215    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.884271 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884244    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.884870 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884853    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.884937 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884885    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.884937 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884908    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.885012 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884949    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.885012 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884979    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.885012 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.884991    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.887204 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.887190    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.887246 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.887215    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 19 14:46:38.887937 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.887922    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientMemory"
Jul 19 14:46:38.887993 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.887957    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasNoDiskPressure"
Jul 19 14:46:38.887993 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.887974    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeHasSufficientPID"
Jul 19 14:46:38.894227 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.894210    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:38.903439 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.903424    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0629444ec59d540f7d4f04814af2104d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal\" (UID: \"0629444ec59d540f7d4f04814af2104d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.903504 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.903447    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0629444ec59d540f7d4f04814af2104d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal\" (UID: \"0629444ec59d540f7d4f04814af2104d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.903504 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:38.903464    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/52a2fdb785bd958c3f182967e497262f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-98.ec2.internal\" (UID: \"52a2fdb785bd958c3f182967e497262f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.908890 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.908876    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-98.ec2.internal\" not found" node="ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.912299 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.912284    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-98.ec2.internal\" not found" node="ip-10-0-135-98.ec2.internal"
Jul 19 14:46:38.994795 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:38.994778    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.004067 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.004047    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0629444ec59d540f7d4f04814af2104d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal\" (UID: \"0629444ec59d540f7d4f04814af2104d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.004116 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.004075    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0629444ec59d540f7d4f04814af2104d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal\" (UID: \"0629444ec59d540f7d4f04814af2104d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.004116 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.004094    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/52a2fdb785bd958c3f182967e497262f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-98.ec2.internal\" (UID: \"52a2fdb785bd958c3f182967e497262f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.004179 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.004140    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0629444ec59d540f7d4f04814af2104d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal\" (UID: \"0629444ec59d540f7d4f04814af2104d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.004179 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.004149    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/52a2fdb785bd958c3f182967e497262f-config\") pod \"kube-apiserver-proxy-ip-10-0-135-98.ec2.internal\" (UID: \"52a2fdb785bd958c3f182967e497262f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.004179 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.004172    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0629444ec59d540f7d4f04814af2104d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal\" (UID: \"0629444ec59d540f7d4f04814af2104d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.095378 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.095348    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.196139 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.196067    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.213532 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.213512    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.216668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.216652    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:39.296619 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.296584    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.397124 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.397078    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.497696 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.497617    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.554030 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.553941    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:39.597780 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.597753    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.636479 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.636452    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 19 14:46:39.636911 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.636585    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:46:39.636911 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.636610    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:46:39.636911 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.636617    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 19 14:46:39.698232 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.698197    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.700830 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.700769    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-18 14:41:39 +0000 UTC" deadline="2027-12-15 22:54:23.316019366 +0000 UTC"
Jul 19 14:46:39.700830 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.700823    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12344h7m43.615201045s"
Jul 19 14:46:39.792242 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:39.792215    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a2fdb785bd958c3f182967e497262f.slice/crio-710ff96dbb31ef31a483d75d28d09c18e422f1a52fcb0ae043971f0e2a866df5 WatchSource:0}: Error finding container 710ff96dbb31ef31a483d75d28d09c18e422f1a52fcb0ae043971f0e2a866df5: Status 404 returned error can't find the container with id 710ff96dbb31ef31a483d75d28d09c18e422f1a52fcb0ae043971f0e2a866df5
Jul 19 14:46:39.792584 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:39.792565    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0629444ec59d540f7d4f04814af2104d.slice/crio-a2cbd63b98c2942677bebca67b0103cd7741ba41f970cf1394c052e094fd130b WatchSource:0}: Error finding container a2cbd63b98c2942677bebca67b0103cd7741ba41f970cf1394c052e094fd130b: Status 404 returned error can't find the container with id a2cbd63b98c2942677bebca67b0103cd7741ba41f970cf1394c052e094fd130b
Jul 19 14:46:39.798764 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.798745    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.798819 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.798803    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:46:39.899888 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:39.899864    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-98.ec2.internal\" not found"
Jul 19 14:46:39.970071 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:39.970042    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:40.002147 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.002124    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:40.009759 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.009744    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:46:40.011033 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.011020    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal"
Jul 19 14:46:40.020005 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.019985    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 19 14:46:40.679890 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.679853    2572 apiserver.go:52] "Watching apiserver"
Jul 19 14:46:40.688847 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.688818    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 19 14:46:40.690917 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.690502    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-xz7z7","openshift-dns/node-resolver-cchfz","openshift-network-diagnostics/network-check-target-rll2r","openshift-network-operator/iptables-alerter-llqqv","openshift-ovn-kubernetes/ovnkube-node-wfm6z","kube-system/global-pull-secret-syncer-gwmb8","kube-system/konnectivity-agent-8fx5w","openshift-image-registry/node-ca-n9985","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal","openshift-multus/multus-additional-cni-plugins-4wwcf","openshift-multus/multus-d2ghn","openshift-multus/network-metrics-daemon-rkxt7","kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"]
Jul 19 14:46:40.693372 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.693334    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.695871 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.695847    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.696227 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.696205    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-55krb\""
Jul 19 14:46:40.696680 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.696550    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 19 14:46:40.696680 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.696566    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 19 14:46:40.699311 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.698256    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:40.699311 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.698357    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:40.699311 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.698770    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8z8f8\""
Jul 19 14:46:40.699311 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.698998    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.699311 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.699174    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 19 14:46:40.699618 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.699578    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 19 14:46:40.699708 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.699689    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.700609 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.700591    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.702930 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.702910    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 19 14:46:40.703242 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.703221    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5pjq4\""
Jul 19 14:46:40.703534 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.703505    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.703676 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.703654    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.704042 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.704021    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 19 14:46:40.704295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.704272    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.705856 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.705837    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.705987 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.705904    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.707336    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.707370    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.707501    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.707560    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.707809    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.708027    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bm5qs\""
Jul 19 14:46:40.708437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.708247    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 19 14:46:40.710037 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.710017    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.712888 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.712861    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.713453    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.714096    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.714312    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.714654    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8tv67\""
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.714920    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3dbd6e4-0932-4dfb-b293-9db3588d47c0-konnectivity-ca\") pod \"konnectivity-agent-8fx5w\" (UID: \"b3dbd6e4-0932-4dfb-b293-9db3588d47c0\") " pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.714946    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.714983    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-ovn\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-cni-bin\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715160    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovnkube-script-lib\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715200    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q647j\" (UniqueName: \"kubernetes.io/projected/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-kube-api-access-q647j\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715240    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-netns\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715292    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-etc-kubernetes\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czvvk\" (UniqueName: \"kubernetes.io/projected/0bcaf722-89e8-42aa-aad9-42788288a317-kube-api-access-czvvk\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-etc-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715397    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715474    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-os-release\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715532    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-run-netns\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovnkube-config\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715634    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-hostroot\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-multus-certs\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715713    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715748    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-slash\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.715816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715776    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-run-ovn-kubernetes\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715836    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-kubelet\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.716399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3dbd6e4-0932-4dfb-b293-9db3588d47c0-agent-certs\") pod \"konnectivity-agent-8fx5w\" (UID: \"b3dbd6e4-0932-4dfb-b293-9db3588d47c0\") " pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.716399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.715905    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-systemd\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-log-socket\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-cni-netd\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716108    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-system-cni-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.716727 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bcaf722-89e8-42aa-aad9-42788288a317-iptables-alerter-script\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.716727 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716463    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.716727 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716484    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-systemd-units\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716727 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716521    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-var-lib-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716914 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716560    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-env-overrides\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.716957 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716912    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-cnibin\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.716957 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716946    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-cni-bin\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717064 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.716990    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bcaf722-89e8-42aa-aad9-42788288a317-host-slash\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.717064 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717022    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-node-log\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.717171 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717054    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovn-node-metrics-cert\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.717171 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717134    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-cni-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717276 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421cc822-b22a-4418-88f2-d5492e93bdc7-cni-binary-copy\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717276 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717221    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.717376 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717327    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-socket-dir-parent\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717471 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717405    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-conf-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717471 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-daemon-config\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717573 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717469    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 19 14:46:40.717573 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717505    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hh2kr\""
Jul 19 14:46:40.717573 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717474    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-kubelet\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.717718 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717573    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.717718 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-k8s-cni-cncf-io\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.717718 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.717641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-cni-multus\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.718249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.718227    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p425l\" (UniqueName: \"kubernetes.io/projected/421cc822-b22a-4418-88f2-d5492e93bdc7-kube-api-access-p425l\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.718438 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.718401    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 19 14:46:40.719748 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.719003    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.719748 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.719037    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.719748 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.719201    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-248vh\""
Jul 19 14:46:40.719748 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.719723    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.722155 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.722134    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-chv22\""
Jul 19 14:46:40.722495 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.722478    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 19 14:46:40.722592 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.722550    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 19 14:46:40.722741 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.722724    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.723863 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.723841    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:40.725747 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.725034    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:40.725747 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.725113    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:40.725747 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.725205    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2whms\""
Jul 19 14:46:40.725747 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.725453    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:46:40.725747 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.725507    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 19 14:46:40.737650 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.737627    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4mf8f"
Jul 19 14:46:40.744583 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.744564    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4mf8f"
Jul 19 14:46:40.784501 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.784452    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal" event={"ID":"0629444ec59d540f7d4f04814af2104d","Type":"ContainerStarted","Data":"a2cbd63b98c2942677bebca67b0103cd7741ba41f970cf1394c052e094fd130b"}
Jul 19 14:46:40.786403 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.786372    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal" event={"ID":"52a2fdb785bd958c3f182967e497262f","Type":"ContainerStarted","Data":"710ff96dbb31ef31a483d75d28d09c18e422f1a52fcb0ae043971f0e2a866df5"}
Jul 19 14:46:40.804121 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.804097    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 19 14:46:40.818756 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818733    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-run-netns\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.818864 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818761    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovnkube-config\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.818864 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818790    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-hostroot\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.818864 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-multus-certs\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.818864 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818844    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818871    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-run-netns\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/388682d3-450c-43ac-8c2d-448bf2d98b9d-hosts-file\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818899    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-multus-certs\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818880    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-hostroot\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818924    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-slash\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.818963    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-tuned\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819004    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-slash\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819007    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3dbd6e4-0932-4dfb-b293-9db3588d47c0-agent-certs\") pod \"konnectivity-agent-8fx5w\" (UID: \"b3dbd6e4-0932-4dfb-b293-9db3588d47c0\") " pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819053    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-systemd\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.819085 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819075    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-system-cni-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.819644 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819103    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.819644 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819124    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-kubernetes\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.819644 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819134    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-systemd\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.819644 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819365    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-var-lib-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.826575 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.826537    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 19 14:46:40.826988 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819194    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-system-cni-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.827179 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.827163    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovnkube-config\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.819256    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-var-lib-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.827936    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-env-overrides\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.827961    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-cnibin\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.827982    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-socket-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828030    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828064    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-cnibin\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bcaf722-89e8-42aa-aad9-42788288a317-host-slash\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828132    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-cni-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-socket-dir-parent\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828178    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-conf-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828181    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bcaf722-89e8-42aa-aad9-42788288a317-host-slash\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828194    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-daemon-config\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828212    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-registration-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828232    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-kubelet-config\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.828580 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828240    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-conf-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828256    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-kubelet\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-cni-multus\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828313    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-sys\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828337    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3dbd6e4-0932-4dfb-b293-9db3588d47c0-konnectivity-ca\") pod \"konnectivity-agent-8fx5w\" (UID: \"b3dbd6e4-0932-4dfb-b293-9db3588d47c0\") " pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828339    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-cni-dir\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828404    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-cni-bin\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828408    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-cni-multus\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828415    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-socket-dir-parent\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-netns\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828403    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-env-overrides\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828471    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-etc-kubernetes\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-sys-fs\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828506    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-netns\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmhrn\" (UniqueName: \"kubernetes.io/projected/469e1276-145e-4db9-be68-a6335d724c20-kube-api-access-mmhrn\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828547    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-kubelet\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828557    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-etc-kubernetes\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.829486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828609    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828643    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a092a032-b5e0-4112-8106-b4201294fc7b-host\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828650    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-cni-bin\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-etc-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828698    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828725    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-device-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828752    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-os-release\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828778    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysconfig\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828782    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828724    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-etc-openvswitch\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828803    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-var-lib-kubelet\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828804    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/421cc822-b22a-4418-88f2-d5492e93bdc7-multus-daemon-config\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828827    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-host\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828859    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69d5dc16-babd-445d-b24b-52d27a66f17e-tmp\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828927    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828967    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-cnibin\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.828987    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3dbd6e4-0932-4dfb-b293-9db3588d47c0-konnectivity-ca\") pod \"konnectivity-agent-8fx5w\" (UID: \"b3dbd6e4-0932-4dfb-b293-9db3588d47c0\") " pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.830424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvlt7\" (UniqueName: \"kubernetes.io/projected/388682d3-450c-43ac-8c2d-448bf2d98b9d-kube-api-access-rvlt7\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829204    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv24v\" (UniqueName: \"kubernetes.io/projected/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-kube-api-access-fv24v\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829245    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-run-ovn-kubernetes\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829286    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-kubelet\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-run-ovn-kubernetes\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829319    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-lib-modules\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829340    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-kubelet\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829394    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-dbus\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-log-socket\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829458    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-cni-netd\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829488    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k55wh\" (UniqueName: \"kubernetes.io/projected/a092a032-b5e0-4112-8106-b4201294fc7b-kube-api-access-k55wh\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829513    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-system-cni-dir\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829546    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bcaf722-89e8-42aa-aad9-42788288a317-iptables-alerter-script\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829602    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-systemd-units\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829627    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-cni-bin\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/388682d3-450c-43ac-8c2d-448bf2d98b9d-tmp-dir\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-node-log\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831314 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829681    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovn-node-metrics-cert\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829704    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421cc822-b22a-4418-88f2-d5492e93bdc7-cni-binary-copy\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829727    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829782    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-k8s-cni-cncf-io\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829810    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p425l\" (UniqueName: \"kubernetes.io/projected/421cc822-b22a-4418-88f2-d5492e93bdc7-kube-api-access-p425l\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829826    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-modprobe-d\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829841    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysctl-d\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-run\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gtcr\" (UniqueName: \"kubernetes.io/projected/69d5dc16-babd-445d-b24b-52d27a66f17e-kube-api-access-8gtcr\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829897    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829929    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-var-lib-cni-bin\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829961    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-node-log\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830238    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-log-socket\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830268    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-host-cni-netd\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830494    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0bcaf722-89e8-42aa-aad9-42788288a317-iptables-alerter-script\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830536    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.829875    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-systemd-units\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.831944 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830645    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-ovn\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830659    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421cc822-b22a-4418-88f2-d5492e93bdc7-cni-binary-copy\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovnkube-script-lib\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830702    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q647j\" (UniqueName: \"kubernetes.io/projected/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-kube-api-access-q647j\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830705    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-host-run-k8s-cni-cncf-io\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830743    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a092a032-b5e0-4112-8106-b4201294fc7b-serviceca\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830763    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysctl-conf\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830781    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czvvk\" (UniqueName: \"kubernetes.io/projected/0bcaf722-89e8-42aa-aad9-42788288a317-kube-api-access-czvvk\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830798    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-os-release\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830833    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvmw7\" (UniqueName: \"kubernetes.io/projected/de5d621a-6d73-4131-a323-f001e1cf14c0-kube-api-access-rvmw7\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830852    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-systemd\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830924    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/421cc822-b22a-4418-88f2-d5492e93bdc7-os-release\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.830952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-run-ovn\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.831399    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovnkube-script-lib\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.832474 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.831692    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3dbd6e4-0932-4dfb-b293-9db3588d47c0-agent-certs\") pod \"konnectivity-agent-8fx5w\" (UID: \"b3dbd6e4-0932-4dfb-b293-9db3588d47c0\") " pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:40.832878 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.832640    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-ovn-node-metrics-cert\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.835481 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.835461    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:46:40.835481 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.835485    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:46:40.835644 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.835498    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:40.835644 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.835592    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:46:41.335569201 +0000 UTC m=+3.037017252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:40.839347 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.839292    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czvvk\" (UniqueName: \"kubernetes.io/projected/0bcaf722-89e8-42aa-aad9-42788288a317-kube-api-access-czvvk\") pod \"iptables-alerter-llqqv\" (UID: \"0bcaf722-89e8-42aa-aad9-42788288a317\") " pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:40.839454 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.839421    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p425l\" (UniqueName: \"kubernetes.io/projected/421cc822-b22a-4418-88f2-d5492e93bdc7-kube-api-access-p425l\") pod \"multus-d2ghn\" (UID: \"421cc822-b22a-4418-88f2-d5492e93bdc7\") " pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:40.839532 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.839514    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q647j\" (UniqueName: \"kubernetes.io/projected/25c3a9a1-c4a8-4705-a9bc-12b106f29cfd-kube-api-access-q647j\") pod \"ovnkube-node-wfm6z\" (UID: \"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd\") " pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:40.931274 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931191    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k55wh\" (UniqueName: \"kubernetes.io/projected/a092a032-b5e0-4112-8106-b4201294fc7b-kube-api-access-k55wh\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.931274 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931230    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-system-cni-dir\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.931274 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931260    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/388682d3-450c-43ac-8c2d-448bf2d98b9d-tmp-dir\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931311    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-modprobe-d\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931328    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysctl-d\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931326    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-system-cni-dir\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-run\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931404    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gtcr\" (UniqueName: \"kubernetes.io/projected/69d5dc16-babd-445d-b24b-52d27a66f17e-kube-api-access-8gtcr\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931424    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931420    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-run\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931447    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931474    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a092a032-b5e0-4112-8106-b4201294fc7b-serviceca\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysctl-conf\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931510    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvmw7\" (UniqueName: \"kubernetes.io/projected/de5d621a-6d73-4131-a323-f001e1cf14c0-kube-api-access-rvmw7\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-systemd\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.931529    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:40.931561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931550    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931575    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/388682d3-450c-43ac-8c2d-448bf2d98b9d-hosts-file\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.931603    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:46:41.431583397 +0000 UTC m=+3.133031453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931623    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/388682d3-450c-43ac-8c2d-448bf2d98b9d-hosts-file\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931635    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-tuned\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931652    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/388682d3-450c-43ac-8c2d-448bf2d98b9d-tmp-dir\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931657    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931678    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-kubernetes\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.931689    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931708    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:40.931726    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:46:41.431714367 +0000 UTC m=+3.133162400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931729    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-kubernetes\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-socket-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.931838    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-socket-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932137    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932236    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysctl-conf\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932309    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysctl-d\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932318    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-registration-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932357    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-registration-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932139    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a092a032-b5e0-4112-8106-b4201294fc7b-serviceca\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932370    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-systemd\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932384    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-kubelet-config\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932416    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-sys\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932448    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-sys-fs\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932475    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmhrn\" (UniqueName: \"kubernetes.io/projected/469e1276-145e-4db9-be68-a6335d724c20-kube-api-access-mmhrn\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932492    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932502    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a092a032-b5e0-4112-8106-b4201294fc7b-host\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932531    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-device-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.932658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932559    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-os-release\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932585    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysconfig\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932618    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-var-lib-kubelet\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932731    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-modprobe-d\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932761    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932789    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-kubelet-config\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932831    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-var-lib-kubelet\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932854    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-sys\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932860    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-host\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69d5dc16-babd-445d-b24b-52d27a66f17e-tmp\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932882    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932940    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a092a032-b5e0-4112-8106-b4201294fc7b-host\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932952    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-cnibin\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.932987    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-device-dir\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvlt7\" (UniqueName: \"kubernetes.io/projected/388682d3-450c-43ac-8c2d-448bf2d98b9d-kube-api-access-rvlt7\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933033    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-sysconfig\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933050    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv24v\" (UniqueName: \"kubernetes.io/projected/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-kube-api-access-fv24v\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:40.933584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933064    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/de5d621a-6d73-4131-a323-f001e1cf14c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933094    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-cnibin\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933136    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-lib-modules\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-dbus\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933256    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-dbus\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933324    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-lib-modules\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933341    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/469e1276-145e-4db9-be68-a6335d724c20-sys-fs\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933398    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69d5dc16-babd-445d-b24b-52d27a66f17e-host\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.934296 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.933416    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de5d621a-6d73-4131-a323-f001e1cf14c0-os-release\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.935254 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.935231    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69d5dc16-babd-445d-b24b-52d27a66f17e-tmp\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.935413 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.935388    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69d5dc16-babd-445d-b24b-52d27a66f17e-etc-tuned\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.941356 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.941309    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k55wh\" (UniqueName: \"kubernetes.io/projected/a092a032-b5e0-4112-8106-b4201294fc7b-kube-api-access-k55wh\") pod \"node-ca-n9985\" (UID: \"a092a032-b5e0-4112-8106-b4201294fc7b\") " pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:40.941771 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.941731    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gtcr\" (UniqueName: \"kubernetes.io/projected/69d5dc16-babd-445d-b24b-52d27a66f17e-kube-api-access-8gtcr\") pod \"tuned-xz7z7\" (UID: \"69d5dc16-babd-445d-b24b-52d27a66f17e\") " pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:40.942242 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.942216    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvmw7\" (UniqueName: \"kubernetes.io/projected/de5d621a-6d73-4131-a323-f001e1cf14c0-kube-api-access-rvmw7\") pod \"multus-additional-cni-plugins-4wwcf\" (UID: \"de5d621a-6d73-4131-a323-f001e1cf14c0\") " pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:40.942324 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.942288    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvlt7\" (UniqueName: \"kubernetes.io/projected/388682d3-450c-43ac-8c2d-448bf2d98b9d-kube-api-access-rvlt7\") pod \"node-resolver-cchfz\" (UID: \"388682d3-450c-43ac-8c2d-448bf2d98b9d\") " pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:40.943073 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.943051    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmhrn\" (UniqueName: \"kubernetes.io/projected/469e1276-145e-4db9-be68-a6335d724c20-kube-api-access-mmhrn\") pod \"aws-ebs-csi-driver-node-fwdfd\" (UID: \"469e1276-145e-4db9-be68-a6335d724c20\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:40.943303 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:40.943281    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv24v\" (UniqueName: \"kubernetes.io/projected/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-kube-api-access-fv24v\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:41.009531 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.009498    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:46:41.023939 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.023916    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2ghn"
Jul 19 14:46:41.033522 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.033500    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-llqqv"
Jul 19 14:46:41.040381 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.040361    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:46:41.048385 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.048359    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd"
Jul 19 14:46:41.056263 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.056243    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n9985"
Jul 19 14:46:41.065194 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.065177    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cchfz"
Jul 19 14:46:41.070981 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.070963    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wwcf"
Jul 19 14:46:41.077739 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.077724    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xz7z7"
Jul 19 14:46:41.156732 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.156703    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:41.336142 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.336094    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:41.336319 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.336307    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:46:41.336422 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.336330    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:46:41.336422 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.336365    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:41.336544 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.336436    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:46:42.336413126 +0000 UTC m=+4.037861176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:41.437020 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.436977    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:41.437208 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.437030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:41.437208 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.437155    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:41.437208 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.437156    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:41.437408 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.437212    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:46:42.437198077 +0000 UTC m=+4.138646111 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:41.437408 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:41.437228    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:46:42.437221418 +0000 UTC m=+4.138669451 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:41.747962 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.747916    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:41:41 +0000 UTC" deadline="2028-01-28 05:56:39.882482103 +0000 UTC"
Jul 19 14:46:41.747962 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.747961    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13383h9m58.134523679s"
Jul 19 14:46:41.756531 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.756507    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcaf722_89e8_42aa_aad9_42788288a317.slice/crio-fb47fb12a527cd85990ef68ff39c74e2a302c22313440f1ae7a5787e5804c195 WatchSource:0}: Error finding container fb47fb12a527cd85990ef68ff39c74e2a302c22313440f1ae7a5787e5804c195: Status 404 returned error can't find the container with id fb47fb12a527cd85990ef68ff39c74e2a302c22313440f1ae7a5787e5804c195
Jul 19 14:46:41.758186 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.758078    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5d621a_6d73_4131_a323_f001e1cf14c0.slice/crio-35c7149a13f9931d72f5174a3da1749b030292071ac6082af20dabbd1cb4b77b WatchSource:0}: Error finding container 35c7149a13f9931d72f5174a3da1749b030292071ac6082af20dabbd1cb4b77b: Status 404 returned error can't find the container with id 35c7149a13f9931d72f5174a3da1749b030292071ac6082af20dabbd1cb4b77b
Jul 19 14:46:41.759094 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.759050    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda092a032_b5e0_4112_8106_b4201294fc7b.slice/crio-00ff4f73a777aee0d824a17addd90ac4f8509dedc46312961a3957545477c15d WatchSource:0}: Error finding container 00ff4f73a777aee0d824a17addd90ac4f8509dedc46312961a3957545477c15d: Status 404 returned error can't find the container with id 00ff4f73a777aee0d824a17addd90ac4f8509dedc46312961a3957545477c15d
Jul 19 14:46:41.759841 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.759815    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421cc822_b22a_4418_88f2_d5492e93bdc7.slice/crio-b31d3b2eeb1eb4a3d0881668b902d7d3edd6e0a0b1ecd4bf9f286765d8708ae3 WatchSource:0}: Error finding container b31d3b2eeb1eb4a3d0881668b902d7d3edd6e0a0b1ecd4bf9f286765d8708ae3: Status 404 returned error can't find the container with id b31d3b2eeb1eb4a3d0881668b902d7d3edd6e0a0b1ecd4bf9f286765d8708ae3
Jul 19 14:46:41.785240 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.785219    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469e1276_145e_4db9_be68_a6335d724c20.slice/crio-bcd7ba676485d057891f1a8c798d03bfe5d19879eee53d48334e2a222602ed66 WatchSource:0}: Error finding container bcd7ba676485d057891f1a8c798d03bfe5d19879eee53d48334e2a222602ed66: Status 404 returned error can't find the container with id bcd7ba676485d057891f1a8c798d03bfe5d19879eee53d48334e2a222602ed66
Jul 19 14:46:41.786287 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.786265    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c3a9a1_c4a8_4705_a9bc_12b106f29cfd.slice/crio-192916a2cc32d2949b20f355f862954d9d05bd4d60156757b33d8750f1f6e737 WatchSource:0}: Error finding container 192916a2cc32d2949b20f355f862954d9d05bd4d60156757b33d8750f1f6e737: Status 404 returned error can't find the container with id 192916a2cc32d2949b20f355f862954d9d05bd4d60156757b33d8750f1f6e737
Jul 19 14:46:41.787150 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:46:41.787056    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod388682d3_450c_43ac_8c2d_448bf2d98b9d.slice/crio-82311e1ec1e4977a534ec50ea7c2278755d259558a97554e3f567c35a67f373c WatchSource:0}: Error finding container 82311e1ec1e4977a534ec50ea7c2278755d259558a97554e3f567c35a67f373c: Status 404 returned error can't find the container with id 82311e1ec1e4977a534ec50ea7c2278755d259558a97554e3f567c35a67f373c
Jul 19 14:46:41.795464 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.795430    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd" event={"ID":"469e1276-145e-4db9-be68-a6335d724c20","Type":"ContainerStarted","Data":"bcd7ba676485d057891f1a8c798d03bfe5d19879eee53d48334e2a222602ed66"}
Jul 19 14:46:41.797548 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.797514    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2ghn" event={"ID":"421cc822-b22a-4418-88f2-d5492e93bdc7","Type":"ContainerStarted","Data":"b31d3b2eeb1eb4a3d0881668b902d7d3edd6e0a0b1ecd4bf9f286765d8708ae3"}
Jul 19 14:46:41.799101 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.799077    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerStarted","Data":"35c7149a13f9931d72f5174a3da1749b030292071ac6082af20dabbd1cb4b77b"}
Jul 19 14:46:41.800666 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.800643    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n9985" event={"ID":"a092a032-b5e0-4112-8106-b4201294fc7b","Type":"ContainerStarted","Data":"00ff4f73a777aee0d824a17addd90ac4f8509dedc46312961a3957545477c15d"}
Jul 19 14:46:41.801824 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:41.801801    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llqqv" event={"ID":"0bcaf722-89e8-42aa-aad9-42788288a317","Type":"ContainerStarted","Data":"fb47fb12a527cd85990ef68ff39c74e2a302c22313440f1ae7a5787e5804c195"}
Jul 19 14:46:42.345231 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.344270    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:42.345231 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.344764    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:46:42.345231 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.344787    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:46:42.345231 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.344803    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:42.345231 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.344865    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:46:44.344845994 +0000 UTC m=+6.046294032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:42.446341 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.445482    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:42.446341 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.445537    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:42.446341 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.445722    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:42.446341 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.445793    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:46:44.445770527 +0000 UTC m=+6.147218586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:42.446341 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.446245    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:42.446341 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.446304    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:46:44.446285197 +0000 UTC m=+6.147733256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:42.714733 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.714634    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 19 14:46:42.748711 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.748642    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-18 14:41:41 +0000 UTC" deadline="2027-12-31 08:25:26.984063642 +0000 UTC"
Jul 19 14:46:42.748711 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.748692    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12713h38m44.235377604s"
Jul 19 14:46:42.782874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.782843    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:42.783058 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.782976    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:42.783426 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.783390    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:42.783706 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.783533    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:42.783706 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.783594    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:42.783706 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:42.783677    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:42.828550 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.825595    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal" event={"ID":"52a2fdb785bd958c3f182967e497262f","Type":"ContainerStarted","Data":"f375b7cefc61793f7a99344bb47cf9bcae615b5b3a31498f4c8126a148b144bd"}
Jul 19 14:46:42.840140 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.840102    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8fx5w" event={"ID":"b3dbd6e4-0932-4dfb-b293-9db3588d47c0","Type":"ContainerStarted","Data":"42c673957c03f83a2134d38bb1cffe8a8358fadcc983ebefc669a82693a9e510"}
Jul 19 14:46:42.841978 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.841933    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-98.ec2.internal" podStartSLOduration=2.841914256 podStartE2EDuration="2.841914256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:42.841046327 +0000 UTC m=+4.542494386" watchObservedRunningTime="2026-07-19 14:46:42.841914256 +0000 UTC m=+4.543362308"
Jul 19 14:46:42.856213 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.856185    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cchfz" event={"ID":"388682d3-450c-43ac-8c2d-448bf2d98b9d","Type":"ContainerStarted","Data":"82311e1ec1e4977a534ec50ea7c2278755d259558a97554e3f567c35a67f373c"}
Jul 19 14:46:42.862231 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.862127    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"192916a2cc32d2949b20f355f862954d9d05bd4d60156757b33d8750f1f6e737"}
Jul 19 14:46:42.878866 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.878528    2572 generic.go:356] "Generic (PLEG): container finished" podID="0629444ec59d540f7d4f04814af2104d" containerID="e3cc87272a3eec39a1967f55bbf40ffc55dd6c350b91a4f2e8334f71c0a2231b" exitCode=0
Jul 19 14:46:42.878866 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.878620    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal" event={"ID":"0629444ec59d540f7d4f04814af2104d","Type":"ContainerDied","Data":"e3cc87272a3eec39a1967f55bbf40ffc55dd6c350b91a4f2e8334f71c0a2231b"}
Jul 19 14:46:42.888039 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:42.887993    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xz7z7" event={"ID":"69d5dc16-babd-445d-b24b-52d27a66f17e","Type":"ContainerStarted","Data":"011fec50550a6e666f162c14fe7e799eed8b5b4a851461d866c9df3475f99aaf"}
Jul 19 14:46:43.914305 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:43.914262    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal" event={"ID":"0629444ec59d540f7d4f04814af2104d","Type":"ContainerStarted","Data":"f9731109d563e24c52033f661ae66a42fe47a8960489e892d5dc2ce91b63c2ca"}
Jul 19 14:46:43.931895 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:43.931827    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-98.ec2.internal" podStartSLOduration=3.931807643 podStartE2EDuration="3.931807643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:46:43.930498417 +0000 UTC m=+5.631946475" watchObservedRunningTime="2026-07-19 14:46:43.931807643 +0000 UTC m=+5.633255706"
Jul 19 14:46:44.360888 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:44.360842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:44.361076 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.361032    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:46:44.361076 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.361056    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:46:44.361076 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.361071    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:44.361263 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.361136    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:46:48.361116589 +0000 UTC m=+10.062564649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:44.462232 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:44.462173    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:44.462232 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:44.462233    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:44.462554 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.462408    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:44.462616 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.462554    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:46:48.462531642 +0000 UTC m=+10.163979680 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:44.463072 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.463013    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:44.463224 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.463195    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:46:48.463175776 +0000 UTC m=+10.164623812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:44.782730 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:44.782644    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:44.782874 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.782802    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:44.782923 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:44.782879    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:44.783596 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.782948    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:44.783596 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:44.783012    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:44.783596 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:44.783061    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:46.780614 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:46.780580    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:46.781088 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:46.780581    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:46.781088 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:46.780695    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:46.781088 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:46.780822    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:46.781088 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:46.780893    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:46.781088 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:46.780980    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:48.394743 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:48.394706    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:48.395251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.394861    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:46:48.395251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.394883    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:46:48.395251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.394891    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:48.395251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.394944    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:46:56.394927764 +0000 UTC m=+18.096375820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:48.495623 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:48.495328    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:48.495623 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:48.495391    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:48.495623 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.495483    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:48.495623 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.495545    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:48.495623 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.495554    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:46:56.495533286 +0000 UTC m=+18.196981322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:48.495623 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.495594    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:46:56.495579211 +0000 UTC m=+18.197027246 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:48.784723 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:48.784272    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:48.784723 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.784417    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:48.784723 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:48.784470    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:48.784723 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.784540    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:48.784723 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:48.784577    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:48.784723 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:48.784640    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:50.782476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:50.781549    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:50.782476 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:50.781699    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:50.782476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:50.782118    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:50.782476 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:50.782212    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:50.782476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:50.782285    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:50.782476 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:50.782381    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:52.780442 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:52.780404    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:52.780857 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:52.780412    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:52.780857 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:52.780508    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:52.780857 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:52.780586    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:52.780857 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:52.780631    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:52.780857 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:52.780705    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:54.780096 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:54.779583    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:54.780096 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:54.779604    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:54.780096 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:54.779604    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:54.780096 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:54.779721    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:54.780096 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:54.779784    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:54.780096 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:54.779864    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:56.462641 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:56.462603    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:56.463189 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.462798    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:46:56.463189 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.462817    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:46:56.463189 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.462829    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:56.463189 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.462884    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:47:12.462866459 +0000 UTC m=+34.164314505 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:46:56.563908 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:56.563868    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:56.564081 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:56.563914    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:56.564081 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.564050    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:56.564193 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.564108    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:47:12.56408988 +0000 UTC m=+34.265537918 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:46:56.564548 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.564521    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:56.564666 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.564574    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:12.564561239 +0000 UTC m=+34.266009270 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:46:56.780175 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:56.780097    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:56.780175 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:56.780143    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:56.780420 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:56.780097    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:56.780420 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.780218    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:56.780420 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.780326    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:56.780548 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:56.780425    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:46:58.780917 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:58.780873    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:46:58.781385 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:58.780997    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:46:58.781385 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:58.781008    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:46:58.781385 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:58.781105    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:46:58.781385 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:46:58.781139    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:46:58.781385 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:46:58.781216    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:00.780426 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:00.780385    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:00.780854 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:00.780385    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:00.780854 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:00.780532    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:00.780854 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:00.780547    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:00.780854 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:00.780659    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:00.780854 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:00.780741    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:01.955138 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.954702    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2ghn" event={"ID":"421cc822-b22a-4418-88f2-d5492e93bdc7","Type":"ContainerStarted","Data":"1c2c285e2f4b8628514c6ac051e0a4c1645cc5428e7e712709ad322667c7ebc6"}
Jul 19 14:47:01.957467 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.957439    2572 generic.go:356] "Generic (PLEG): container finished" podID="de5d621a-6d73-4131-a323-f001e1cf14c0" containerID="3d74726bf2544796a6ea8c3a41dd03a2321502b3dede7504a0b612c10e8e58b2" exitCode=0
Jul 19 14:47:01.957570 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.957529    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerDied","Data":"3d74726bf2544796a6ea8c3a41dd03a2321502b3dede7504a0b612c10e8e58b2"}
Jul 19 14:47:01.961446 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.961420    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n9985" event={"ID":"a092a032-b5e0-4112-8106-b4201294fc7b","Type":"ContainerStarted","Data":"52ff9f479813739fab7dba02ccc58b2a69f7fe585d784daf2d404e3ded5f09f6"}
Jul 19 14:47:01.964400 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.964372    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8fx5w" event={"ID":"b3dbd6e4-0932-4dfb-b293-9db3588d47c0","Type":"ContainerStarted","Data":"52939695314c1aad98e07347e4dba41acba72f1c1b01339b9b020a7a4daf2277"}
Jul 19 14:47:01.966054 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.966032    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cchfz" event={"ID":"388682d3-450c-43ac-8c2d-448bf2d98b9d","Type":"ContainerStarted","Data":"829a2df9d01647e8d05121fa429e29f9eec9610b97a36c1ef2dbb7cf90658203"}
Jul 19 14:47:01.969050 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969034    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:47:01.969529 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969507    2572 generic.go:356] "Generic (PLEG): container finished" podID="25c3a9a1-c4a8-4705-a9bc-12b106f29cfd" containerID="bac1f08645bf8055c41831243eebe97c5f277121a95d72c52852357fedde5656" exitCode=1
Jul 19 14:47:01.969629 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"b14c83ecf2610d7d6c72d409f264f945eaf5eae9c6e39d60296743d7ddbab010"}
Jul 19 14:47:01.969688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"37feb0b558f23b1a0857ececa126dca9285f5be4729d19485191f85ddb624584"}
Jul 19 14:47:01.969743 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969722    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"2ec9cb3b5e5338cfbfb6039380db47862689e991599738cb2c25ae9c3587831f"}
Jul 19 14:47:01.969803 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969786    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerDied","Data":"bac1f08645bf8055c41831243eebe97c5f277121a95d72c52852357fedde5656"}
Jul 19 14:47:01.969862 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.969846    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"6c403d6d17d72578d0f4c626f8efb7013b6e5e29b7e223115d11ba791e3a2615"}
Jul 19 14:47:01.971393 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.971268    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd" event={"ID":"469e1276-145e-4db9-be68-a6335d724c20","Type":"ContainerStarted","Data":"06df1221eb0d8db059f7d5e51a21d62e6f5c3065e4e49a0b2e1127e803f7eed8"}
Jul 19 14:47:01.973161 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.973129    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xz7z7" event={"ID":"69d5dc16-babd-445d-b24b-52d27a66f17e","Type":"ContainerStarted","Data":"e95324bc44000de18761f43dc73dd075f51c0a85f67b40b36a2a255902734d5e"}
Jul 19 14:47:01.977966 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.977922    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d2ghn" podStartSLOduration=3.4090807229999998 podStartE2EDuration="22.977905524s" totalImagesPullingTime="19.568824801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:01.977094178 +0000 UTC m=+23.678542230" watchObservedRunningTime="2026-07-19 14:47:01.977905524 +0000 UTC m=+23.679353579"
Jul 19 14:47:01.996023 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:01.995952    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8fx5w" podStartSLOduration=3.648387429 podStartE2EDuration="22.995934762s" totalImagesPullingTime="19.347547333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:01.995561876 +0000 UTC m=+23.697009931" watchObservedRunningTime="2026-07-19 14:47:01.995934762 +0000 UTC m=+23.697382816"
Jul 19 14:47:02.010746 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.010706    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cchfz" podStartSLOduration=3.584584162 podStartE2EDuration="23.010693691s" totalImagesPullingTime="19.426109529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:02.010396774 +0000 UTC m=+23.711844829" watchObservedRunningTime="2026-07-19 14:47:02.010693691 +0000 UTC m=+23.712141743"
Jul 19 14:47:02.025128 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.025089    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n9985" podStartSLOduration=3.668559857 podStartE2EDuration="23.025075949s" totalImagesPullingTime="19.356516092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:02.024499803 +0000 UTC m=+23.725947858" watchObservedRunningTime="2026-07-19 14:47:02.025075949 +0000 UTC m=+23.726524003"
Jul 19 14:47:02.066633 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.066586    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xz7z7" podStartSLOduration=3.6487177969999998 podStartE2EDuration="23.066571799s" totalImagesPullingTime="19.417854002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:02.06632993 +0000 UTC m=+23.767777984" watchObservedRunningTime="2026-07-19 14:47:02.066571799 +0000 UTC m=+23.768019852"
Jul 19 14:47:02.698223 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.698195    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 19 14:47:02.725608 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.725574    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:47:02.782294 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.782184    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-19T14:47:02.698214833Z","UUID":"43020b0f-0928-4c52-84fd-e89ce9050d4b","Handler":null,"Name":"","Endpoint":""}
Jul 19 14:47:02.783626 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.783597    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:02.783758 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:02.783718    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:02.783850 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.783835    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 19 14:47:02.783892 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.783871    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 19 14:47:02.783984 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.783971    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:02.784023 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.784006    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:02.784077 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:02.784063    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:02.784136 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:02.784119    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:02.978092 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.978055    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd" event={"ID":"469e1276-145e-4db9-be68-a6335d724c20","Type":"ContainerStarted","Data":"adcf065e6e7b5c20e18dba91e6cc5a53422434835ea90b2ee17b2a116e88990f"}
Jul 19 14:47:02.979849 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.979821    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-llqqv" event={"ID":"0bcaf722-89e8-42aa-aad9-42788288a317","Type":"ContainerStarted","Data":"76f363722e2fe30a8925947cf6c9f104b03b194fe2a199b83335fd03ca8c4c86"}
Jul 19 14:47:02.983109 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.983091    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:47:02.983530 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.983504    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"3948953a89b6bf66f69c25f3de4b5cb584106b9875633ada9969b5a620e1681b"}
Jul 19 14:47:02.994841 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:02.994796    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-llqqv" podStartSLOduration=4.540370677 podStartE2EDuration="23.994783428s" totalImagesPullingTime="19.454412751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:02.994249965 +0000 UTC m=+24.695698021" watchObservedRunningTime="2026-07-19 14:47:02.994783428 +0000 UTC m=+24.696231481"
Jul 19 14:47:03.988577 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:03.988539    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd" event={"ID":"469e1276-145e-4db9-be68-a6335d724c20","Type":"ContainerStarted","Data":"bdf2dce33caf010086e1847ad2b6389b5300271e7f04ddb935e78fc1821b89a5"}
Jul 19 14:47:04.006887 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:04.006831    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwdfd" podStartSLOduration=3.407222884 podStartE2EDuration="25.006811206s" totalImagesPullingTime="21.599588322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:04.00638466 +0000 UTC m=+25.707832714" watchObservedRunningTime="2026-07-19 14:47:04.006811206 +0000 UTC m=+25.708259263"
Jul 19 14:47:04.780248 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:04.780215    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:04.780465 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:04.780361    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:04.780722 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:04.780700    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:04.780804 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:04.780789    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:04.780971 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:04.780945    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:04.781099 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:04.781050    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:04.994940 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:04.994911    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:47:04.995406 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:04.995376    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"813d3cea088e0d571608ca2384d390564c9c05327e5414fd841fc3930e1ac836"}
Jul 19 14:47:05.266237 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:05.266204    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:47:05.266840 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:05.266817    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:47:05.998327 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:05.998155    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8fx5w"
Jul 19 14:47:06.780023 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:06.779988    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:06.780023 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:06.780018    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:06.780251 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:06.780018    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:06.780251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:06.780111    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:06.780251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:06.780188    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:06.780251 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:06.780242    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:07.002601 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.002574    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:47:07.003251 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.002946    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"6afbcb7b6539581378b5bdd24f7368be68ad91ad774f8ebf36f2fe072d0648a4"}
Jul 19 14:47:07.003251 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.003188    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:47:07.003373 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.003254    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:47:07.003431 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.003400    2572 scope.go:117] "RemoveContainer" containerID="bac1f08645bf8055c41831243eebe97c5f277121a95d72c52852357fedde5656"
Jul 19 14:47:07.005146 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.005128    2572 generic.go:356] "Generic (PLEG): container finished" podID="de5d621a-6d73-4131-a323-f001e1cf14c0" containerID="0ebd903d92844f805ba8087156e023ba2fb1ff5205de08af06d877f9fd1fd182" exitCode=0
Jul 19 14:47:07.005257 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.005216    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerDied","Data":"0ebd903d92844f805ba8087156e023ba2fb1ff5205de08af06d877f9fd1fd182"}
Jul 19 14:47:07.019618 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:07.019598    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:47:08.010011 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.009844    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:47:08.010477 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.010382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" event={"ID":"25c3a9a1-c4a8-4705-a9bc-12b106f29cfd","Type":"ContainerStarted","Data":"a4cb359b37d3a338876836be36d6ead6de49ab8e71a59527ee26b712deb9cdea"}
Jul 19 14:47:08.010527 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.010487    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:47:08.025980 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.025960    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:47:08.035154 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.035116    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z" podStartSLOduration=9.48280403 podStartE2EDuration="29.035102022s" totalImagesPullingTime="19.552297992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:08.033591652 +0000 UTC m=+29.735039707" watchObservedRunningTime="2026-07-19 14:47:08.035102022 +0000 UTC m=+29.736550078"
Jul 19 14:47:08.558488 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.558419    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rll2r"]
Jul 19 14:47:08.558670 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.558522    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:08.558670 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:08.558601    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:08.561139 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.561111    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gwmb8"]
Jul 19 14:47:08.561268 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.561218    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:08.561386 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:08.561319    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:08.561849 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.561826    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkxt7"]
Jul 19 14:47:08.561956 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:08.561938    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:08.562047 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:08.562026    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:09.017117 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:09.016779    2572 generic.go:356] "Generic (PLEG): container finished" podID="de5d621a-6d73-4131-a323-f001e1cf14c0" containerID="a8bf1e48853071a7fcad89c110dd9bc31b083d2924e63baae35f09fd40ca659a" exitCode=0
Jul 19 14:47:09.017681 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:09.017167    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerDied","Data":"a8bf1e48853071a7fcad89c110dd9bc31b083d2924e63baae35f09fd40ca659a"}
Jul 19 14:47:09.780610 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:09.780582    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:09.780790 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:09.780682    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:10.779781 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:10.779624    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:10.779781 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:10.779641    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:10.779781 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:10.779747    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:10.780327 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:10.779905    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:11.024682 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:11.024530    2572 generic.go:356] "Generic (PLEG): container finished" podID="de5d621a-6d73-4131-a323-f001e1cf14c0" containerID="64f220f89aecfd9cf15b0ef7d0a45e4eb8467138808b9cb910bc681f67f9d810" exitCode=0
Jul 19 14:47:11.024787 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:11.024590    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerDied","Data":"64f220f89aecfd9cf15b0ef7d0a45e4eb8467138808b9cb910bc681f67f9d810"}
Jul 19 14:47:11.777117 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:11.777081    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:11.780002 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:11.779974    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:11.780381 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:11.780101    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:11.865550 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:11.865521    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:12.487586 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:12.487552    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:12.487773 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.487732    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 19 14:47:12.487773 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.487750    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 19 14:47:12.487773 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.487760    2572 projected.go:196] Error preparing data for projected volume kube-api-access-kwc5c for pod openshift-network-diagnostics/network-check-target-rll2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:12.487901 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.487821    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c podName:8de18eb1-c33d-4e85-af45-80b91e64e1da nodeName:}" failed. No retries permitted until 2026-07-19 14:47:44.487806301 +0000 UTC m=+66.189254335 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-kwc5c" (UniqueName: "kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c") pod "network-check-target-rll2r" (UID: "8de18eb1-c33d-4e85-af45-80b91e64e1da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 19 14:47:12.588693 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:12.588660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:12.588876 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:12.588712    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:12.588876 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.588786    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:12.588876 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.588816    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:12.588876 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.588850    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:47:44.588832328 +0000 UTC m=+66.290280380 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 19 14:47:12.588876 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.588870    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret podName:d965c61d-cbd0-4f16-bc14-7d87a547cbfe nodeName:}" failed. No retries permitted until 2026-07-19 14:47:44.588861915 +0000 UTC m=+66.290309950 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret") pod "global-pull-secret-syncer-gwmb8" (UID: "d965c61d-cbd0-4f16-bc14-7d87a547cbfe") : object "kube-system"/"original-pull-secret" not registered
Jul 19 14:47:12.780645 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:12.780555    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:12.781062 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.780687    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:47:12.781062 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:12.780753    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:12.781062 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:12.780870    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rll2r" podUID="8de18eb1-c33d-4e85-af45-80b91e64e1da"
Jul 19 14:47:13.779602 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:13.779569    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:13.779784 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:13.779704    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gwmb8" podUID="d965c61d-cbd0-4f16-bc14-7d87a547cbfe"
Jul 19 14:47:13.941072 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:13.941032    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.001536 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.001492    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.097350 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.097318    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-98.ec2.internal" event="NodeReady"
Jul 19 14:47:14.097538 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.097488    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 19 14:47:14.142748 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.142714    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hzq6p"]
Jul 19 14:47:14.145462 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.145441    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.145993 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.145974    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-24nxl"]
Jul 19 14:47:14.147498 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.147483    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.147952 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.147934    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 19 14:47:14.148045 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.147942    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 19 14:47:14.148353 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.148323    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4r258\""
Jul 19 14:47:14.150401 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.150245    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 19 14:47:14.150583 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.150564    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 19 14:47:14.150704 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.150567    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q8987\""
Jul 19 14:47:14.150976 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.150957    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 19 14:47:14.155612 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.155596    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hzq6p"]
Jul 19 14:47:14.158980 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.158959    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-24nxl"]
Jul 19 14:47:14.267045 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.267009    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.301045 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.301008    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/926fed00-bf12-4e4d-a042-9811b951d1fc-config-volume\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.301209 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.301084    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3f0b971-6e80-4db5-8008-77916f998dfe-cert\") pod \"ingress-canary-24nxl\" (UID: \"d3f0b971-6e80-4db5-8008-77916f998dfe\") " pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.301209 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.301113    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.301209 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.301134    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824jg\" (UniqueName: \"kubernetes.io/projected/926fed00-bf12-4e4d-a042-9811b951d1fc-kube-api-access-824jg\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.301209 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.301166    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b84mw\" (UniqueName: \"kubernetes.io/projected/d3f0b971-6e80-4db5-8008-77916f998dfe-kube-api-access-b84mw\") pod \"ingress-canary-24nxl\" (UID: \"d3f0b971-6e80-4db5-8008-77916f998dfe\") " pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.301209 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.301188    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926fed00-bf12-4e4d-a042-9811b951d1fc-tmp-dir\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.368691 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.368612    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.401623 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.401582    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.401803 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.401632    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824jg\" (UniqueName: \"kubernetes.io/projected/926fed00-bf12-4e4d-a042-9811b951d1fc-kube-api-access-824jg\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.401803 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.401670    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b84mw\" (UniqueName: \"kubernetes.io/projected/d3f0b971-6e80-4db5-8008-77916f998dfe-kube-api-access-b84mw\") pod \"ingress-canary-24nxl\" (UID: \"d3f0b971-6e80-4db5-8008-77916f998dfe\") " pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.401803 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.401696    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926fed00-bf12-4e4d-a042-9811b951d1fc-tmp-dir\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.401803 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:14.401747    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:14.401803 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.401756    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/926fed00-bf12-4e4d-a042-9811b951d1fc-config-volume\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.402049 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.401812    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3f0b971-6e80-4db5-8008-77916f998dfe-cert\") pod \"ingress-canary-24nxl\" (UID: \"d3f0b971-6e80-4db5-8008-77916f998dfe\") " pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.402049 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:14.401833    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:47:14.901809812 +0000 UTC m=+36.603257858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:14.402547 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.402514    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/926fed00-bf12-4e4d-a042-9811b951d1fc-tmp-dir\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.402893 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.402870    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/926fed00-bf12-4e4d-a042-9811b951d1fc-config-volume\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.406912 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.406876    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d3f0b971-6e80-4db5-8008-77916f998dfe-cert\") pod \"ingress-canary-24nxl\" (UID: \"d3f0b971-6e80-4db5-8008-77916f998dfe\") " pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.412794 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.412767    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b84mw\" (UniqueName: \"kubernetes.io/projected/d3f0b971-6e80-4db5-8008-77916f998dfe-kube-api-access-b84mw\") pod \"ingress-canary-24nxl\" (UID: \"d3f0b971-6e80-4db5-8008-77916f998dfe\") " pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.412881 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.412823    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-824jg\" (UniqueName: \"kubernetes.io/projected/926fed00-bf12-4e4d-a042-9811b951d1fc-kube-api-access-824jg\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.465006 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.464968    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-24nxl"
Jul 19 14:47:14.780189 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.780112    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:14.780526 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.780508    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:14.782939 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.782919    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:47:14.783038 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.782921    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:47:14.784132 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.784113    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c6kqk\""
Jul 19 14:47:14.784238 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.784147    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5p5rq\""
Jul 19 14:47:14.784299 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.784262    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:47:14.905377 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.905321    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:14.905576 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:14.905499    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:14.905636 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:14.905591    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:47:15.905570856 +0000 UTC m=+37.607018894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:14.917495 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.917467    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:14.979626 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:14.979598    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:15.263974 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:15.263787    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-24nxl"]
Jul 19 14:47:15.268156 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:47:15.268131    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f0b971_6e80_4db5_8008_77916f998dfe.slice/crio-87751a218858c57c8824a5c247837a8eff62c8daa383b6e46952b0076a7cc309 WatchSource:0}: Error finding container 87751a218858c57c8824a5c247837a8eff62c8daa383b6e46952b0076a7cc309: Status 404 returned error can't find the container with id 87751a218858c57c8824a5c247837a8eff62c8daa383b6e46952b0076a7cc309
Jul 19 14:47:15.780304 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:15.780270    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:15.782965 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:15.782948    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:47:15.914007 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:15.913968    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:15.914164 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:15.914103    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:15.914232 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:15.914173    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:47:17.914153808 +0000 UTC m=+39.615601842 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:16.039248 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:16.039165    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-24nxl" event={"ID":"d3f0b971-6e80-4db5-8008-77916f998dfe","Type":"ContainerStarted","Data":"87751a218858c57c8824a5c247837a8eff62c8daa383b6e46952b0076a7cc309"}
Jul 19 14:47:16.042230 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:16.042207    2572 generic.go:356] "Generic (PLEG): container finished" podID="de5d621a-6d73-4131-a323-f001e1cf14c0" containerID="c46ead0b8147223b2e9e5b8ecfac4b98a950ea7801f55fa368c5892ca0e4b240" exitCode=0
Jul 19 14:47:16.042323 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:16.042245    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerDied","Data":"c46ead0b8147223b2e9e5b8ecfac4b98a950ea7801f55fa368c5892ca0e4b240"}
Jul 19 14:47:16.072382 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:16.072352    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:16.132747 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:16.132720    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:16.234512 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:16.234482    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:17.049449 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:17.049418    2572 generic.go:356] "Generic (PLEG): container finished" podID="de5d621a-6d73-4131-a323-f001e1cf14c0" containerID="bf69945eaeaad346a73b5a1ca16c60f87f842adeb4e76e48246a31d3b6db46f9" exitCode=0
Jul 19 14:47:17.049885 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:17.049486    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerDied","Data":"bf69945eaeaad346a73b5a1ca16c60f87f842adeb4e76e48246a31d3b6db46f9"}
Jul 19 14:47:17.929128 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:17.929053    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:17.929263 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:17.929170    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:17.929263 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:17.929220    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:47:21.92920722 +0000 UTC m=+43.630655252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:18.054871 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:18.054842    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" event={"ID":"de5d621a-6d73-4131-a323-f001e1cf14c0","Type":"ContainerStarted","Data":"9fed808cab52ca4f381d0a7c88c4e04b2d5342e538b80f1d8e8c9f25fe844bc2"}
Jul 19 14:47:18.056166 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:18.056145    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-24nxl" event={"ID":"d3f0b971-6e80-4db5-8008-77916f998dfe","Type":"ContainerStarted","Data":"f88b2ce7db90eb7537b323a9fd8d1a134171e36f9c0a01157111ac41ac551e92"}
Jul 19 14:47:18.082650 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:18.082613    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4wwcf" podStartSLOduration=8.951196841 podStartE2EDuration="39.08259804s" totalImagesPullingTime="30.131401199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:18.081085275 +0000 UTC m=+39.782533329" watchObservedRunningTime="2026-07-19 14:47:18.08259804 +0000 UTC m=+39.784046121"
Jul 19 14:47:18.098613 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:18.098569    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-24nxl" podStartSLOduration=1.8126625440000002 podStartE2EDuration="4.098549414s" totalImagesPullingTime="2.28588687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:18.097983961 +0000 UTC m=+39.799432014" watchObservedRunningTime="2026-07-19 14:47:18.098549414 +0000 UTC m=+39.799997466"
Jul 19 14:47:18.226404 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:18.226304    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:18.330489 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:18.330289    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:19.724172 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:19.724134    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log" limit=1024
Jul 19 14:47:21.960183 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:21.960146    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:21.960669 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:21.960326    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:21.960669 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:21.960471    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:47:29.960445973 +0000 UTC m=+51.661894008 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:30.014358 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:30.014301    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:30.014897 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:30.014460    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:30.014897 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:30.014546    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:47:46.014524534 +0000 UTC m=+67.715972567 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:40.036183 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:40.036155    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wfm6z"
Jul 19 14:47:44.510694 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.510656    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:44.513507 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.513486    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 19 14:47:44.524469 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.524449    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 19 14:47:44.535862 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.535841    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwc5c\" (UniqueName: \"kubernetes.io/projected/8de18eb1-c33d-4e85-af45-80b91e64e1da-kube-api-access-kwc5c\") pod \"network-check-target-rll2r\" (UID: \"8de18eb1-c33d-4e85-af45-80b91e64e1da\") " pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:44.611124 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.611079    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:47:44.611124 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.611130    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:44.613975 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.613955    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 19 14:47:44.614028 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.613955    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 19 14:47:44.622243 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:44.622225    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:47:44.622344 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:44.622302    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:48:48.622284097 +0000 UTC m=+130.323732148 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : secret "metrics-daemon-secret" not found
Jul 19 14:47:44.624505 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.624482    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d965c61d-cbd0-4f16-bc14-7d87a547cbfe-original-pull-secret\") pod \"global-pull-secret-syncer-gwmb8\" (UID: \"d965c61d-cbd0-4f16-bc14-7d87a547cbfe\") " pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:44.794753 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.794676    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c6kqk\""
Jul 19 14:47:44.802953 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.802934    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:44.889596 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.889571    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gwmb8"
Jul 19 14:47:44.924140 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:44.924111    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rll2r"]
Jul 19 14:47:44.928048 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:47:44.928019    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8de18eb1_c33d_4e85_af45_80b91e64e1da.slice/crio-9c379185a3799d07c18291f643e6384fdef61bbf128b5d9582421f1a1d2ed6f2 WatchSource:0}: Error finding container 9c379185a3799d07c18291f643e6384fdef61bbf128b5d9582421f1a1d2ed6f2: Status 404 returned error can't find the container with id 9c379185a3799d07c18291f643e6384fdef61bbf128b5d9582421f1a1d2ed6f2
Jul 19 14:47:45.010356 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:45.010315    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gwmb8"]
Jul 19 14:47:45.013127 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:47:45.013103    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd965c61d_cbd0_4f16_bc14_7d87a547cbfe.slice/crio-c3915df8645acb5b51bafe31b4db6bbb79e1ebee16032a108df734502c1c31dd WatchSource:0}: Error finding container c3915df8645acb5b51bafe31b4db6bbb79e1ebee16032a108df734502c1c31dd: Status 404 returned error can't find the container with id c3915df8645acb5b51bafe31b4db6bbb79e1ebee16032a108df734502c1c31dd
Jul 19 14:47:45.120512 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:45.120469    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gwmb8" event={"ID":"d965c61d-cbd0-4f16-bc14-7d87a547cbfe","Type":"ContainerStarted","Data":"c3915df8645acb5b51bafe31b4db6bbb79e1ebee16032a108df734502c1c31dd"}
Jul 19 14:47:45.121622 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:45.121597    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rll2r" event={"ID":"8de18eb1-c33d-4e85-af45-80b91e64e1da","Type":"ContainerStarted","Data":"9c379185a3799d07c18291f643e6384fdef61bbf128b5d9582421f1a1d2ed6f2"}
Jul 19 14:47:46.020956 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:46.020921    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:47:46.021397 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:46.021061    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:47:46.021397 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:47:46.021127    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:48:18.021105472 +0000 UTC m=+99.722553528 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:47:50.136348 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:50.136299    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gwmb8" event={"ID":"d965c61d-cbd0-4f16-bc14-7d87a547cbfe","Type":"ContainerStarted","Data":"0ae3fad8506a0b24aaf3a81d0b35555d53105af09dec0f881489c8b1b7c23f2a"}
Jul 19 14:47:50.137754 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:50.137726    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rll2r" event={"ID":"8de18eb1-c33d-4e85-af45-80b91e64e1da","Type":"ContainerStarted","Data":"53b47df900ecbd872adf4288fb8466fe63073cee49fa5ef0066a56344bffd988"}
Jul 19 14:47:50.137878 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:50.137778    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:47:50.152822 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:50.152776    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gwmb8" podStartSLOduration=66.770177163 podStartE2EDuration="1m11.152764533s" totalImagesPullingTime="4.38258737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:50.151655821 +0000 UTC m=+71.853103877" watchObservedRunningTime="2026-07-19 14:47:50.152764533 +0000 UTC m=+71.854212587"
Jul 19 14:47:50.166550 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:47:50.166516    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rll2r" podStartSLOduration=66.702269372 podStartE2EDuration="1m11.166506877s" totalImagesPullingTime="4.464237505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:47:50.166066376 +0000 UTC m=+71.867514430" watchObservedRunningTime="2026-07-19 14:47:50.166506877 +0000 UTC m=+71.867954931"
Jul 19 14:48:07.440420 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.440251    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"]
Jul 19 14:48:07.444434 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.444418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.447708 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.447688    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 19 14:48:07.449159 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.449100    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 19 14:48:07.449159 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.449111    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 19 14:48:07.449159 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.449111    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 19 14:48:07.455559 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.455539    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"]
Jul 19 14:48:07.554963 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.554932    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.554963 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.554963    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-tmp\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.555166 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.554989    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpth7\" (UniqueName: \"kubernetes.io/projected/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-kube-api-access-dpth7\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.655639 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.655610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.655639 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.655639    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-tmp\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.655838 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.655660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpth7\" (UniqueName: \"kubernetes.io/projected/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-kube-api-access-dpth7\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.656007 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.655985    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-tmp\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.658219 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.658196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.663643 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.663623    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpth7\" (UniqueName: \"kubernetes.io/projected/a0c7bab8-13bf-4995-8713-dcd5209ef5b4-kube-api-access-dpth7\") pod \"klusterlet-addon-workmgr-7dddddfcf7-krbpd\" (UID: \"a0c7bab8-13bf-4995-8713-dcd5209ef5b4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.753057 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.752980    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:07.872147 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:07.872023    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"]
Jul 19 14:48:07.874759 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:48:07.874734    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c7bab8_13bf_4995_8713_dcd5209ef5b4.slice/crio-4fa94a11b669b9c73053aa87e0ab39147b96b99195bb83dc2850f79927f42612 WatchSource:0}: Error finding container 4fa94a11b669b9c73053aa87e0ab39147b96b99195bb83dc2850f79927f42612: Status 404 returned error can't find the container with id 4fa94a11b669b9c73053aa87e0ab39147b96b99195bb83dc2850f79927f42612
Jul 19 14:48:08.179354 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:08.179308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd" event={"ID":"a0c7bab8-13bf-4995-8713-dcd5209ef5b4","Type":"ContainerStarted","Data":"4fa94a11b669b9c73053aa87e0ab39147b96b99195bb83dc2850f79927f42612"}
Jul 19 14:48:12.191459 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:12.191423    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd" event={"ID":"a0c7bab8-13bf-4995-8713-dcd5209ef5b4","Type":"ContainerStarted","Data":"e24f1499abd2cb67c29139c29d5af7c240e9ca638d0dc61c118ce03db4097a11"}
Jul 19 14:48:12.191940 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:12.191647    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:12.193055 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:12.193033    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:48:12.206706 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:12.206670    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd" podStartSLOduration=1.122615769 podStartE2EDuration="5.206657417s" totalImagesPullingTime="4.084041648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:48:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:48:12.206522385 +0000 UTC m=+93.907970442" watchObservedRunningTime="2026-07-19 14:48:12.206657417 +0000 UTC m=+93.908105472"
Jul 19 14:48:18.024919 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:18.024883    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:48:18.025404 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:48:18.025027    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:48:18.025404 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:48:18.025105    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:49:22.025086 +0000 UTC m=+163.726534043 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:48:21.142547 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:21.142519    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rll2r"
Jul 19 14:48:48.627839 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:48:48.627796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:48:48.628357 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:48:48.627946    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:48:48.628357 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:48:48.628011    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:50:50.627995832 +0000 UTC m=+252.329443864 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : secret "metrics-daemon-secret" not found
Jul 19 14:49:17.157721 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:49:17.157684    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hzq6p" podUID="926fed00-bf12-4e4d-a042-9811b951d1fc"
Jul 19 14:49:17.346913 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:49:17.346886    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:49:17.797950 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:49:17.797890    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:49:22.046463 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:49:22.046434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:49:22.046929 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:49:22.046610    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 19 14:49:22.046929 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:49:22.046718    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls podName:926fed00-bf12-4e4d-a042-9811b951d1fc nodeName:}" failed. No retries permitted until 2026-07-19 14:51:24.046695146 +0000 UTC m=+285.748143177 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls") pod "dns-default-hzq6p" (UID: "926fed00-bf12-4e4d-a042-9811b951d1fc") : secret "dns-default-metrics-tls" not found
Jul 19 14:49:32.779989 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:49:32.779885    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:50:12.192405 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:12.192358    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd" podUID="a0c7bab8-13bf-4995-8713-dcd5209ef5b4" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.8:8000/readyz\": dial tcp 10.133.0.8:8000: connect: connection refused"
Jul 19 14:50:12.477543 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:12.477511    2572 generic.go:356] "Generic (PLEG): container finished" podID="a0c7bab8-13bf-4995-8713-dcd5209ef5b4" containerID="e24f1499abd2cb67c29139c29d5af7c240e9ca638d0dc61c118ce03db4097a11" exitCode=1
Jul 19 14:50:12.477702 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:12.477552    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd" event={"ID":"a0c7bab8-13bf-4995-8713-dcd5209ef5b4","Type":"ContainerDied","Data":"e24f1499abd2cb67c29139c29d5af7c240e9ca638d0dc61c118ce03db4097a11"}
Jul 19 14:50:12.477832 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:12.477820    2572 scope.go:117] "RemoveContainer" containerID="e24f1499abd2cb67c29139c29d5af7c240e9ca638d0dc61c118ce03db4097a11"
Jul 19 14:50:13.484575 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:13.484541    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd" event={"ID":"a0c7bab8-13bf-4995-8713-dcd5209ef5b4","Type":"ContainerStarted","Data":"93579f6f6ab2f4c96b51e4f7256bcec4e5f8d731f8808ea1c46a669eaf8e2f04"}
Jul 19 14:50:13.484973 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:13.484831    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:50:13.486133 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:13.486107    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dddddfcf7-krbpd"
Jul 19 14:50:50.718886 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:50:50.718848    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:50:50.719429 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:50:50.718965    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 19 14:50:50.719429 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:50:50.719019    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs podName:36ff5be0-b9b5-46b8-a4d3-02765a4a6331 nodeName:}" failed. No retries permitted until 2026-07-19 14:52:52.719005347 +0000 UTC m=+374.420453378 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs") pod "network-metrics-daemon-rkxt7" (UID: "36ff5be0-b9b5-46b8-a4d3-02765a4a6331") : secret "metrics-daemon-secret" not found
Jul 19 14:51:01.943880 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.943806    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"]
Jul 19 14:51:01.946564 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.946548    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"]
Jul 19 14:51:01.946684 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.946664    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:01.949163 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.949143    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 19 14:51:01.949307 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.949174    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 19 14:51:01.949307 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.949214    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 19 14:51:01.949307 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.949287    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c56786b4-222cc"]
Jul 19 14:51:01.949445 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.949410    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:01.951707 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.951693    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84f985cd78-z6vnc"]
Jul 19 14:51:01.951804 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.951759    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:01.952221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.952198    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:51:01.952484 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.952470    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7smxl\""
Jul 19 14:51:01.954404 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.954388    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.955092 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955073    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 19 14:51:01.955217 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955190    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 19 14:51:01.955322 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955259    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bjg94\""
Jul 19 14:51:01.955524 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955509    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 19 14:51:01.955609 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955572    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 19 14:51:01.955609 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955592    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 19 14:51:01.955738 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955611    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 19 14:51:01.955738 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.955652    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 19 14:51:01.956361 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.956345    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 19 14:51:01.956456 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.956418    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-hfp2f\""
Jul 19 14:51:01.956456 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.956368    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 19 14:51:01.956564 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.956349    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 19 14:51:01.957850 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.957827    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 19 14:51:01.958650 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.958629    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"]
Jul 19 14:51:01.959127 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.959105    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 19 14:51:01.961905 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.961867    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 19 14:51:01.962042 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.961914    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bbjc4\""
Jul 19 14:51:01.964950 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.964913    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 19 14:51:01.965077 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.965041    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"]
Jul 19 14:51:01.965843 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.965825    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f985cd78-z6vnc"]
Jul 19 14:51:01.977823 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.977802    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c56786b4-222cc"]
Jul 19 14:51:01.995553 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995533    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-installation-pull-secrets\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995567    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995588    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a4f6541-420b-4d0a-841f-4903afb44f58-ca-trust-extracted\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995617    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bttvh\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-kube-api-access-bttvh\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995644    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8hff\" (UniqueName: \"kubernetes.io/projected/bec9c880-4f12-4177-aac1-f6474b24054c-kube-api-access-f8hff\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995662    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-stats-auth\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995688    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-image-registry-private-configuration\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995710    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995737    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bec9c880-4f12-4177-aac1-f6474b24054c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995767    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bec9c880-4f12-4177-aac1-f6474b24054c-config\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995804    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-trusted-ca\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995827    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-default-certificate\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995849    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-bound-sa-token\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.995874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995870    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:01.996221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995905    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-certificates\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:01.996221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995925    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:01.996221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995945    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pszxj\" (UniqueName: \"kubernetes.io/projected/590bf656-2b4e-473e-bbb0-9d73010f86ad-kube-api-access-pszxj\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:01.996221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:01.995962    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7whtr\" (UniqueName: \"kubernetes.io/projected/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-kube-api-access-7whtr\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.097082 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097029    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.097082 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097087    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a4f6541-420b-4d0a-841f-4903afb44f58-ca-trust-extracted\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097126    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bttvh\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-kube-api-access-bttvh\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8hff\" (UniqueName: \"kubernetes.io/projected/bec9c880-4f12-4177-aac1-f6474b24054c-kube-api-access-f8hff\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097188    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-stats-auth\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097224    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-image-registry-private-configuration\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097283    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bec9c880-4f12-4177-aac1-f6474b24054c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097306    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bec9c880-4f12-4177-aac1-f6474b24054c-config\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.097371 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097353    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097381    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-trusted-ca\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097411    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-default-certificate\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.097189    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.097566    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f985cd78-z6vnc: secret "image-registry-tls" not found
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a4f6541-420b-4d0a-841f-4903afb44f58-ca-trust-extracted\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.097677    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.097455    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:02.5974244 +0000 UTC m=+264.298872444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : configmap references non-existent config key: service-ca.crt
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.097751    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls podName:0a4f6541-420b-4d0a-841f-4903afb44f58 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:02.597727514 +0000 UTC m=+264.299175551 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls") pod "image-registry-84f985cd78-z6vnc" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58") : secret "image-registry-tls" not found
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.097772    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls podName:b30b8fc1-74ae-409b-b60b-ddd88fd659bb nodeName:}" failed. No retries permitted until 2026-07-19 14:51:02.597762832 +0000 UTC m=+264.299210864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-f2csq" (UID: "b30b8fc1-74ae-409b-b60b-ddd88fd659bb") : secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:02.097817 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097798    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-bound-sa-token\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097830    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097871    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-certificates\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097906    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097934    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pszxj\" (UniqueName: \"kubernetes.io/projected/590bf656-2b4e-473e-bbb0-9d73010f86ad-kube-api-access-pszxj\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.097939    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bec9c880-4f12-4177-aac1-f6474b24054c-config\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.098082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7whtr\" (UniqueName: \"kubernetes.io/projected/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-kube-api-access-7whtr\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.098510 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.098171    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-installation-pull-secrets\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.098946 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.098629    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-certificates\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.098946 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.098718    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 19 14:51:02.098946 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.098766    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:02.598749024 +0000 UTC m=+264.300197070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : secret "router-metrics-certs-default" not found
Jul 19 14:51:02.098946 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.098940    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-trusted-ca\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.099541 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.099518    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.100010 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.099985    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bec9c880-4f12-4177-aac1-f6474b24054c-serving-cert\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.100302 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.100281    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-default-certificate\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.100428 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.100406    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-stats-auth\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.100478 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.100436    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-image-registry-private-configuration\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.100969 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.100950    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-installation-pull-secrets\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.106422 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.106393    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8hff\" (UniqueName: \"kubernetes.io/projected/bec9c880-4f12-4177-aac1-f6474b24054c-kube-api-access-f8hff\") pod \"service-ca-operator-5bcbc94c78-ls8bd\" (UID: \"bec9c880-4f12-4177-aac1-f6474b24054c\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.106839 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.106817    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bttvh\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-kube-api-access-bttvh\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.107707 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.107679    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pszxj\" (UniqueName: \"kubernetes.io/projected/590bf656-2b4e-473e-bbb0-9d73010f86ad-kube-api-access-pszxj\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.108065 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.108045    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-bound-sa-token\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.108239 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.108218    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7whtr\" (UniqueName: \"kubernetes.io/projected/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-kube-api-access-7whtr\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.257155 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.257090    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"
Jul 19 14:51:02.381283 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.381233    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd"]
Jul 19 14:51:02.384117 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:02.384093    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec9c880_4f12_4177_aac1_f6474b24054c.slice/crio-845faf1f0be35ad46a28d4217be884e4821f89197c9753b29e23bd4aab5b9f47 WatchSource:0}: Error finding container 845faf1f0be35ad46a28d4217be884e4821f89197c9753b29e23bd4aab5b9f47: Status 404 returned error can't find the container with id 845faf1f0be35ad46a28d4217be884e4821f89197c9753b29e23bd4aab5b9f47
Jul 19 14:51:02.604377 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.604315    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:02.604552 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.604395    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.604552 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604454    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:02.604552 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604526    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls podName:b30b8fc1-74ae-409b-b60b-ddd88fd659bb nodeName:}" failed. No retries permitted until 2026-07-19 14:51:03.604504713 +0000 UTC m=+265.305952764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-f2csq" (UID: "b30b8fc1-74ae-409b-b60b-ddd88fd659bb") : secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:02.604552 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604540    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 19 14:51:02.604552 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604553    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:51:02.604755 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604565    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f985cd78-z6vnc: secret "image-registry-tls" not found
Jul 19 14:51:02.604755 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604590    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:03.604577914 +0000 UTC m=+265.306025947 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : secret "router-metrics-certs-default" not found
Jul 19 14:51:02.604755 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.604464    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:02.604755 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604611    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls podName:0a4f6541-420b-4d0a-841f-4903afb44f58 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:03.604597362 +0000 UTC m=+265.306045412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls") pod "image-registry-84f985cd78-z6vnc" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58") : secret "image-registry-tls" not found
Jul 19 14:51:02.604755 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.604677    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:02.604755 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:02.604753    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:03.604746348 +0000 UTC m=+265.306194380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : configmap references non-existent config key: service-ca.crt
Jul 19 14:51:02.619562 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:02.619523    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd" event={"ID":"bec9c880-4f12-4177-aac1-f6474b24054c","Type":"ContainerStarted","Data":"845faf1f0be35ad46a28d4217be884e4821f89197c9753b29e23bd4aab5b9f47"}
Jul 19 14:51:03.614476 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:03.614425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:03.614520    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:03.614558    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:03.614589    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614608    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614631    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f985cd78-z6vnc: secret "image-registry-tls" not found
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614685    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614696    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls podName:0a4f6541-420b-4d0a-841f-4903afb44f58 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:05.614675728 +0000 UTC m=+267.316123786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls") pod "image-registry-84f985cd78-z6vnc" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58") : secret "image-registry-tls" not found
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614706    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614715    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:05.614706336 +0000 UTC m=+267.316154370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : configmap references non-existent config key: service-ca.crt
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614740    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls podName:b30b8fc1-74ae-409b-b60b-ddd88fd659bb nodeName:}" failed. No retries permitted until 2026-07-19 14:51:05.614728412 +0000 UTC m=+267.316176454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-f2csq" (UID: "b30b8fc1-74ae-409b-b60b-ddd88fd659bb") : secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:03.614973 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:03.614755    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:05.614747039 +0000 UTC m=+267.316195070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : secret "router-metrics-certs-default" not found
Jul 19 14:51:04.627352 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:04.627307    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd" event={"ID":"bec9c880-4f12-4177-aac1-f6474b24054c","Type":"ContainerStarted","Data":"ef5aa63c841b40a3d8792ef1a04269ed9158c4890bec90627b0c41f8878fab85"}
Jul 19 14:51:04.645029 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:04.644986    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd" podStartSLOduration=1.542695151 podStartE2EDuration="3.644971483s" totalImagesPullingTime="2.102276332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:04.644579719 +0000 UTC m=+266.346027770" watchObservedRunningTime="2026-07-19 14:51:04.644971483 +0000 UTC m=+266.346419537"
Jul 19 14:51:05.633272 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:05.633228    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:05.633291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:05.633312    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:05.633354    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633376    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633394    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f985cd78-z6vnc: secret "image-registry-tls" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633445    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633466    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633464    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls podName:0a4f6541-420b-4d0a-841f-4903afb44f58 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:09.633437815 +0000 UTC m=+271.334885849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls") pod "image-registry-84f985cd78-z6vnc" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58") : secret "image-registry-tls" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633494    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls podName:b30b8fc1-74ae-409b-b60b-ddd88fd659bb nodeName:}" failed. No retries permitted until 2026-07-19 14:51:09.633483813 +0000 UTC m=+271.334931845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-f2csq" (UID: "b30b8fc1-74ae-409b-b60b-ddd88fd659bb") : secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633515    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:09.633499983 +0000 UTC m=+271.334948040 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : secret "router-metrics-certs-default" not found
Jul 19 14:51:05.633668 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:05.633529    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:09.633523221 +0000 UTC m=+271.334971253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : configmap references non-existent config key: service-ca.crt
Jul 19 14:51:06.826521 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.826491    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6xwv7"]
Jul 19 14:51:06.829794 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.829777    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:06.832825 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.832807    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 19 14:51:06.833236 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.833218    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 19 14:51:06.834165 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.834140    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 19 14:51:06.834274 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.834164    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 19 14:51:06.834274 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.834151    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gskh7\""
Jul 19 14:51:06.845177 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.845157    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xwv7"]
Jul 19 14:51:06.944282 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.944256    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:06.944406 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.944349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:06.944406 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.944374    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt76q\" (UniqueName: \"kubernetes.io/projected/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-api-access-lt76q\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:06.944406 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.944397    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-crio-socket\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:06.944503 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:06.944411    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-data-volume\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045258 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045232    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045467 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045265    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt76q\" (UniqueName: \"kubernetes.io/projected/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-api-access-lt76q\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045467 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045307    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-crio-socket\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045467 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045329    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-data-volume\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045467 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.045412    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:07.045467 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-crio-socket\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045660 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045417    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.045660 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.045476    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.045660 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.045481    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:07.545462424 +0000 UTC m=+269.246910458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:07.045660 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.045545    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:07.545531041 +0000 UTC m=+269.246979072 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.045789 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.045753    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-data-volume\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.053537 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.053511    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt76q\" (UniqueName: \"kubernetes.io/projected/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-api-access-lt76q\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.549044 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.549016    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.549226 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:07.549094    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:07.549226 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.549139    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.549226 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.549188    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:07.549226 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.549201    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:08.549186383 +0000 UTC m=+270.250634415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:07.549226 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:07.549225    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:08.549214233 +0000 UTC m=+270.250662265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:08.557142 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:08.557099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:08.557636 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:08.557201    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:08.557636 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:08.557246    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:08.557636 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:08.557312    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:08.557636 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:08.557330    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:10.557309431 +0000 UTC m=+272.258757466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:08.557636 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:08.557388    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:10.55737216 +0000 UTC m=+272.258820198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:08.637462 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:08.637428    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cchfz_388682d3-450c-43ac-8c2d-448bf2d98b9d/dns-node-resolver/0.log"
Jul 19 14:51:09.666206 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:09.666168    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:09.666232    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:09.666253    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:09.666278    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666371    2572 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666385    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666393    2572 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f985cd78-z6vnc: secret "image-registry-tls" not found
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666397    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:17.666372113 +0000 UTC m=+279.367820155 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : configmap references non-existent config key: service-ca.crt
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666399    2572 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666442    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls podName:b30b8fc1-74ae-409b-b60b-ddd88fd659bb nodeName:}" failed. No retries permitted until 2026-07-19 14:51:17.666424341 +0000 UTC m=+279.367872372 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-f2csq" (UID: "b30b8fc1-74ae-409b-b60b-ddd88fd659bb") : secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666458    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls podName:0a4f6541-420b-4d0a-841f-4903afb44f58 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:17.66645126 +0000 UTC m=+279.367899292 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls") pod "image-registry-84f985cd78-z6vnc" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58") : secret "image-registry-tls" not found
Jul 19 14:51:09.666653 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:09.666467    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:17.666462544 +0000 UTC m=+279.367910575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : secret "router-metrics-certs-default" not found
Jul 19 14:51:09.837769 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:09.837737    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n9985_a092a032-b5e0-4112-8106-b4201294fc7b/node-ca/0.log"
Jul 19 14:51:10.437737 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:10.437710    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-24nxl_d3f0b971-6e80-4db5-8008-77916f998dfe/serve-healthcheck-canary/0.log"
Jul 19 14:51:10.572881 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:10.572842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:10.573042 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:10.572936    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:10.573042 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:10.572964    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:10.573042 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:10.573025    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:14.573010135 +0000 UTC m=+276.274458166 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:10.573042 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:10.573033    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:10.573177 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:10.573093    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:14.573075723 +0000 UTC m=+276.274523768 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:14.606097 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:14.606056    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:14.606518 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:14.606167    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:14.606518 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:14.606190    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 19 14:51:14.606518 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:14.606252    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:22.606233081 +0000 UTC m=+284.307681136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : configmap "kube-rbac-proxy" not found
Jul 19 14:51:14.606518 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:14.606275    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:14.606518 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:14.606309    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls podName:81b36509-ec8b-4c2a-a0ac-165c8c817a7b nodeName:}" failed. No retries permitted until 2026-07-19 14:51:22.606299182 +0000 UTC m=+284.307747219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6xwv7" (UID: "81b36509-ec8b-4c2a-a0ac-165c8c817a7b") : secret "insights-runtime-extractor-tls" not found
Jul 19 14:51:17.731251 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.731206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:17.731751 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.731289    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:17.731751 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.731350    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:17.731751 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.731381    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:17.731751 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:17.731483    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:17.731751 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:17.731499    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle podName:590bf656-2b4e-473e-bbb0-9d73010f86ad nodeName:}" failed. No retries permitted until 2026-07-19 14:51:33.731484504 +0000 UTC m=+295.432932538 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle") pod "router-default-6c56786b4-222cc" (UID: "590bf656-2b4e-473e-bbb0-9d73010f86ad") : configmap references non-existent config key: service-ca.crt
Jul 19 14:51:17.731751 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:17.731541    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls podName:b30b8fc1-74ae-409b-b60b-ddd88fd659bb nodeName:}" failed. No retries permitted until 2026-07-19 14:51:33.731522695 +0000 UTC m=+295.432970735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-f2csq" (UID: "b30b8fc1-74ae-409b-b60b-ddd88fd659bb") : secret "cluster-monitoring-operator-tls" not found
Jul 19 14:51:17.733729 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.733704    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/590bf656-2b4e-473e-bbb0-9d73010f86ad-metrics-certs\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:17.733807 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.733776    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"image-registry-84f985cd78-z6vnc\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") " pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:17.876532 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:17.876489    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:18.004128 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:18.004103    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f985cd78-z6vnc"]
Jul 19 14:51:18.005965 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:18.005935    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4f6541_420b_4d0a_841f_4903afb44f58.slice/crio-fab81d535adaed429f5aba2ee4d656c552125a4749558a919f20dbcda6f03bd5 WatchSource:0}: Error finding container fab81d535adaed429f5aba2ee4d656c552125a4749558a919f20dbcda6f03bd5: Status 404 returned error can't find the container with id fab81d535adaed429f5aba2ee4d656c552125a4749558a919f20dbcda6f03bd5
Jul 19 14:51:18.669143 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:18.669108    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc" event={"ID":"0a4f6541-420b-4d0a-841f-4903afb44f58","Type":"ContainerStarted","Data":"dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800"}
Jul 19 14:51:18.669317 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:18.669164    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc" event={"ID":"0a4f6541-420b-4d0a-841f-4903afb44f58","Type":"ContainerStarted","Data":"fab81d535adaed429f5aba2ee4d656c552125a4749558a919f20dbcda6f03bd5"}
Jul 19 14:51:18.669317 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:18.669237    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:18.688722 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:18.688667    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc" podStartSLOduration=17.688649226 podStartE2EDuration="17.688649226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:18.687626874 +0000 UTC m=+280.389074930" watchObservedRunningTime="2026-07-19 14:51:18.688649226 +0000 UTC m=+280.390097280"
Jul 19 14:51:20.347564 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:20.347501    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-hzq6p" podUID="926fed00-bf12-4e4d-a042-9811b951d1fc"
Jul 19 14:51:20.675094 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:20.675013    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:51:22.673877 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:22.673836    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:22.674529 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:22.673930    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:22.674607 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:22.674568    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:22.676294 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:22.676277    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/81b36509-ec8b-4c2a-a0ac-165c8c817a7b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6xwv7\" (UID: \"81b36509-ec8b-4c2a-a0ac-165c8c817a7b\") " pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:22.738274 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:22.738223    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6xwv7"
Jul 19 14:51:22.860582 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:22.860553    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6xwv7"]
Jul 19 14:51:22.863624 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:22.863592    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b36509_ec8b_4c2a_a0ac_165c8c817a7b.slice/crio-78fe299972eb7f5e5df32123b34227148134de321a9b1569016ba4578f66e796 WatchSource:0}: Error finding container 78fe299972eb7f5e5df32123b34227148134de321a9b1569016ba4578f66e796: Status 404 returned error can't find the container with id 78fe299972eb7f5e5df32123b34227148134de321a9b1569016ba4578f66e796
Jul 19 14:51:23.685724 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:23.685687    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwv7" event={"ID":"81b36509-ec8b-4c2a-a0ac-165c8c817a7b","Type":"ContainerStarted","Data":"78fe299972eb7f5e5df32123b34227148134de321a9b1569016ba4578f66e796"}
Jul 19 14:51:24.085209 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:24.085176    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:51:24.087890 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:24.087868    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/926fed00-bf12-4e4d-a042-9811b951d1fc-metrics-tls\") pod \"dns-default-hzq6p\" (UID: \"926fed00-bf12-4e4d-a042-9811b951d1fc\") " pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:51:24.279080 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:24.279048    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4r258\""
Jul 19 14:51:24.286582 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:24.286551    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:51:24.433894 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:24.433859    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hzq6p"]
Jul 19 14:51:24.437480 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:24.437440    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926fed00_bf12_4e4d_a042_9811b951d1fc.slice/crio-3a53dcd5b7c60f295f40d7d4568d876f4c7d1d564469441a0b3ddf200118c686 WatchSource:0}: Error finding container 3a53dcd5b7c60f295f40d7d4568d876f4c7d1d564469441a0b3ddf200118c686: Status 404 returned error can't find the container with id 3a53dcd5b7c60f295f40d7d4568d876f4c7d1d564469441a0b3ddf200118c686
Jul 19 14:51:24.690619 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:24.690533    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzq6p" event={"ID":"926fed00-bf12-4e4d-a042-9811b951d1fc","Type":"ContainerStarted","Data":"3a53dcd5b7c60f295f40d7d4568d876f4c7d1d564469441a0b3ddf200118c686"}
Jul 19 14:51:28.705919 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:28.705884    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwv7" event={"ID":"81b36509-ec8b-4c2a-a0ac-165c8c817a7b","Type":"ContainerStarted","Data":"b0b1860870e6a312a691be2bbf19ebc65563a526c901e2fbe0abe23ab19c7c5b"}
Jul 19 14:51:28.707768 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:28.707734    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzq6p" event={"ID":"926fed00-bf12-4e4d-a042-9811b951d1fc","Type":"ContainerStarted","Data":"1e17f51e8ba715a750a831a7db111b7e1ca44eb2ac80b190bcebd48657c06e10"}
Jul 19 14:51:28.707896 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:28.707782    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hzq6p" event={"ID":"926fed00-bf12-4e4d-a042-9811b951d1fc","Type":"ContainerStarted","Data":"7d492413c9057e6187d71d362c5210af55e27aef8b550623c6aea412d4f48b81"}
Jul 19 14:51:28.707896 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:28.707833    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:51:28.724322 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:28.724266    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hzq6p" podStartSLOduration=250.909372743 podStartE2EDuration="4m14.724247091s" totalImagesPullingTime="3.814874348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:47:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:28.7231932 +0000 UTC m=+290.424641255" watchObservedRunningTime="2026-07-19 14:51:28.724247091 +0000 UTC m=+290.425695126"
Jul 19 14:51:30.410442 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.410404    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f985cd78-z6vnc"]
Jul 19 14:51:30.447220 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.447189    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"]
Jul 19 14:51:30.463060 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.463029    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.467813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.467777    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"]
Jul 19 14:51:30.639587 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639559    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86cc9cad-cdd1-45fc-857d-d5961be30d0c-trusted-ca\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639757 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639593    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-bound-sa-token\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639757 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639622    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86cc9cad-cdd1-45fc-857d-d5961be30d0c-ca-trust-extracted\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639757 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639644    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-registry-tls\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639757 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639742    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86cc9cad-cdd1-45fc-857d-d5961be30d0c-registry-certificates\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639949 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639781    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86cc9cad-cdd1-45fc-857d-d5961be30d0c-installation-pull-secrets\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639949 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639816    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-599n9\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-kube-api-access-599n9\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.639949 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.639910    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86cc9cad-cdd1-45fc-857d-d5961be30d0c-image-registry-private-configuration\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740590 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740514    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86cc9cad-cdd1-45fc-857d-d5961be30d0c-image-registry-private-configuration\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740602    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86cc9cad-cdd1-45fc-857d-d5961be30d0c-trusted-ca\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740621    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-bound-sa-token\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86cc9cad-cdd1-45fc-857d-d5961be30d0c-ca-trust-extracted\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740656    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-registry-tls\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740680    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86cc9cad-cdd1-45fc-857d-d5961be30d0c-registry-certificates\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740706    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86cc9cad-cdd1-45fc-857d-d5961be30d0c-installation-pull-secrets\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.740770 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.740730    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-599n9\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-kube-api-access-599n9\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.741143 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.741070    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/86cc9cad-cdd1-45fc-857d-d5961be30d0c-ca-trust-extracted\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.741651 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.741628    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/86cc9cad-cdd1-45fc-857d-d5961be30d0c-registry-certificates\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.741777 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.741650    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86cc9cad-cdd1-45fc-857d-d5961be30d0c-trusted-ca\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.743833 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.743806    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/86cc9cad-cdd1-45fc-857d-d5961be30d0c-image-registry-private-configuration\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.743967 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.743899    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/86cc9cad-cdd1-45fc-857d-d5961be30d0c-installation-pull-secrets\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.744130 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.744111    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-registry-tls\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.748978 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.748956    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-bound-sa-token\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.749710 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.749689    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-599n9\" (UniqueName: \"kubernetes.io/projected/86cc9cad-cdd1-45fc-857d-d5961be30d0c-kube-api-access-599n9\") pod \"image-registry-66c5bfdfc4-8bwrm\" (UID: \"86cc9cad-cdd1-45fc-857d-d5961be30d0c\") " pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:30.775784 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:30.775755    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:31.773316 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:31.771291    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"]
Jul 19 14:51:31.777419 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:31.777389    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86cc9cad_cdd1_45fc_857d_d5961be30d0c.slice/crio-eb16e43cd6f8b16750d89b24417e2d3ed091463b21c263b46dffca291ac034c9 WatchSource:0}: Error finding container eb16e43cd6f8b16750d89b24417e2d3ed091463b21c263b46dffca291ac034c9: Status 404 returned error can't find the container with id eb16e43cd6f8b16750d89b24417e2d3ed091463b21c263b46dffca291ac034c9
Jul 19 14:51:32.723683 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:32.723637    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwv7" event={"ID":"81b36509-ec8b-4c2a-a0ac-165c8c817a7b","Type":"ContainerStarted","Data":"77fb6fcf26a71302e83f37c79f1c7a939049b997cc0b2352c9632c60b483e897"}
Jul 19 14:51:32.725195 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:32.725163    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm" event={"ID":"86cc9cad-cdd1-45fc-857d-d5961be30d0c","Type":"ContainerStarted","Data":"4543e10dbf5686223f694270aa1908513dc0bff0a96cde404c037e0db9cb0cc4"}
Jul 19 14:51:32.725315 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:32.725209    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm" event={"ID":"86cc9cad-cdd1-45fc-857d-d5961be30d0c","Type":"ContainerStarted","Data":"eb16e43cd6f8b16750d89b24417e2d3ed091463b21c263b46dffca291ac034c9"}
Jul 19 14:51:32.725315 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:32.725307    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:32.745174 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:32.745119    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm" podStartSLOduration=2.745104784 podStartE2EDuration="2.745104784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:32.744513784 +0000 UTC m=+294.445961839" watchObservedRunningTime="2026-07-19 14:51:32.745104784 +0000 UTC m=+294.446552837"
Jul 19 14:51:33.768511 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:33.768475    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:33.768957 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:33.768523    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:33.769112 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:33.769086    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/590bf656-2b4e-473e-bbb0-9d73010f86ad-service-ca-bundle\") pod \"router-default-6c56786b4-222cc\" (UID: \"590bf656-2b4e-473e-bbb0-9d73010f86ad\") " pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:33.771151 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:33.771129    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:33.771591 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:33.771569    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b30b8fc1-74ae-409b-b60b-ddd88fd659bb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-f2csq\" (UID: \"b30b8fc1-74ae-409b-b60b-ddd88fd659bb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:34.064473 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:34.064386    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"
Jul 19 14:51:35.217500 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.217425    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c56786b4-222cc"]
Jul 19 14:51:35.220926 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:35.220896    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590bf656_2b4e_473e_bbb0_9d73010f86ad.slice/crio-9cb04b503043710d60669557845314dc47e859898354964988f4aa85d037cf8d WatchSource:0}: Error finding container 9cb04b503043710d60669557845314dc47e859898354964988f4aa85d037cf8d: Status 404 returned error can't find the container with id 9cb04b503043710d60669557845314dc47e859898354964988f4aa85d037cf8d
Jul 19 14:51:35.229356 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.229312    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq"]
Jul 19 14:51:35.232168 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:35.232127    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30b8fc1_74ae_409b_b60b_ddd88fd659bb.slice/crio-f5651ee987b706ff773c8024a8e72eb315652ec1b0677839020dfecd0541be8f WatchSource:0}: Error finding container f5651ee987b706ff773c8024a8e72eb315652ec1b0677839020dfecd0541be8f: Status 404 returned error can't find the container with id f5651ee987b706ff773c8024a8e72eb315652ec1b0677839020dfecd0541be8f
Jul 19 14:51:35.737752 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.737718    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c56786b4-222cc" event={"ID":"590bf656-2b4e-473e-bbb0-9d73010f86ad","Type":"ContainerStarted","Data":"3db044361239355161b31eea48b018da11bcb76d398cb66ce7f1f02c54fed4a9"}
Jul 19 14:51:35.737955 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.737773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c56786b4-222cc" event={"ID":"590bf656-2b4e-473e-bbb0-9d73010f86ad","Type":"ContainerStarted","Data":"9cb04b503043710d60669557845314dc47e859898354964988f4aa85d037cf8d"}
Jul 19 14:51:35.738818 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.738797    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq" event={"ID":"b30b8fc1-74ae-409b-b60b-ddd88fd659bb","Type":"ContainerStarted","Data":"f5651ee987b706ff773c8024a8e72eb315652ec1b0677839020dfecd0541be8f"}
Jul 19 14:51:35.740627 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.740606    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6xwv7" event={"ID":"81b36509-ec8b-4c2a-a0ac-165c8c817a7b","Type":"ContainerStarted","Data":"9c1ddc47e08cabce67b36d005f710603fb8dea468120f0b5bc3b431bde8ec9b4"}
Jul 19 14:51:35.757700 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.757653    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c56786b4-222cc" podStartSLOduration=34.757638685 podStartE2EDuration="34.757638685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:35.755840016 +0000 UTC m=+297.457288071" watchObservedRunningTime="2026-07-19 14:51:35.757638685 +0000 UTC m=+297.459086767"
Jul 19 14:51:35.771352 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.771308    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:35.771767 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.771702    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6xwv7" podStartSLOduration=17.62281095 podStartE2EDuration="29.771686441s" totalImagesPullingTime="12.148875491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 14:51:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:35.771600507 +0000 UTC m=+297.473048562" watchObservedRunningTime="2026-07-19 14:51:35.771686441 +0000 UTC m=+297.473134502"
Jul 19 14:51:35.773529 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:35.773511    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:35.780737 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:35.780715    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rkxt7" podUID="36ff5be0-b9b5-46b8-a4d3-02765a4a6331"
Jul 19 14:51:36.745200 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:36.745149    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:36.746563 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:36.746539    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c56786b4-222cc"
Jul 19 14:51:37.749528 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:37.749484    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq" event={"ID":"b30b8fc1-74ae-409b-b60b-ddd88fd659bb","Type":"ContainerStarted","Data":"4575dd1aca49b40df02cd20e23e6d4b0e7c40b0a3fab7156bcc70b492ea79de6"}
Jul 19 14:51:37.767378 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:37.767319    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-f2csq" podStartSLOduration=34.800706434 podStartE2EDuration="36.767305577s" totalImagesPullingTime="1.966599143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:37.766143539 +0000 UTC m=+299.467591594" watchObservedRunningTime="2026-07-19 14:51:37.767305577 +0000 UTC m=+299.468753658"
Jul 19 14:51:38.711894 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:38.711867    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:51:38.713399 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:38.713378    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hzq6p"
Jul 19 14:51:38.716405 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:38.716386    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:51:38.718313 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:38.718281    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 19 14:51:40.416832 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:40.416804    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:49.779700 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:49.779670    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:51:53.124232 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.124201    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"]
Jul 19 14:51:53.127546 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.127531    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.130435 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.130407    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 19 14:51:53.130548 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.130471    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 19 14:51:53.131014 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.130991    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 19 14:51:53.132854 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.132832    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-shlqk\""
Jul 19 14:51:53.141237 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.141216    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"]
Jul 19 14:51:53.148356 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.148322    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"]
Jul 19 14:51:53.151197 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.151181    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.155412 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.155390    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 19 14:51:53.155647 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.155631    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 19 14:51:53.155781 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.155765    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 19 14:51:53.155861 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.155847    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-b655m\""
Jul 19 14:51:53.156874 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.156855    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q8q2g"]
Jul 19 14:51:53.160229 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.160211    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.162768 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.162751    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 19 14:51:53.162768 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.162762    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 19 14:51:53.163319 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.163304    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 19 14:51:53.163421 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.163396    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tfpqx\""
Jul 19 14:51:53.169468 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.169441    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"]
Jul 19 14:51:53.313006 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.312975    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.313006 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313017    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltpxg\" (UniqueName: \"kubernetes.io/projected/8281eeb4-bee7-4399-83c8-0e02a863793b-kube-api-access-ltpxg\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313102    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313138    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ed8ce068-4b87-45cf-84aa-440fd4405a49-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.313249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313159    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-tls\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313177    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.313249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313199    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkl84\" (UniqueName: \"kubernetes.io/projected/8eef126c-9281-452b-b1f6-f81bd1fba198-kube-api-access-dkl84\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.313249 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313225    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jvj7\" (UniqueName: \"kubernetes.io/projected/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-api-access-2jvj7\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313311    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313388    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8281eeb4-bee7-4399-83c8-0e02a863793b-metrics-client-ca\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313425    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313473    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-root\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313500    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313523    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eef126c-9281-452b-b1f6-f81bd1fba198-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.313568 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313554    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-textfile\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313839 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.313839 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313633    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-wtmp\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.313839 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313661    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed8ce068-4b87-45cf-84aa-440fd4405a49-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.313839 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.313686    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-sys\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.414520 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414450    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.414520 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414482    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ed8ce068-4b87-45cf-84aa-440fd4405a49-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.414520 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414500    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-tls\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.414782 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414524    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.414782 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkl84\" (UniqueName: \"kubernetes.io/projected/8eef126c-9281-452b-b1f6-f81bd1fba198-kube-api-access-dkl84\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.414782 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414693    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jvj7\" (UniqueName: \"kubernetes.io/projected/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-api-access-2jvj7\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.414782 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414737    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.414782 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414775    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8281eeb4-bee7-4399-83c8-0e02a863793b-metrics-client-ca\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414808    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414837    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-root\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414862    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414880    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ed8ce068-4b87-45cf-84aa-440fd4405a49-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414885    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eef126c-9281-452b-b1f6-f81bd1fba198-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414946    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-textfile\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.414975    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415027    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-root\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415043 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415028    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-wtmp\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed8ce068-4b87-45cf-84aa-440fd4405a49-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415116    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-sys\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415145    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-wtmp\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415190    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltpxg\" (UniqueName: \"kubernetes.io/projected/8281eeb4-bee7-4399-83c8-0e02a863793b-kube-api-access-ltpxg\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:53.415199    2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415374    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8281eeb4-bee7-4399-83c8-0e02a863793b-metrics-client-ca\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415114    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-accelerators-collector-config\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:53.415444    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls podName:8eef126c-9281-452b-b1f6-f81bd1fba198 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:53.91542269 +0000 UTC m=+315.616870723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-n7ngq" (UID: "8eef126c-9281-452b-b1f6-f81bd1fba198") : secret "openshift-state-metrics-tls" not found
Jul 19 14:51:53.415688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415594    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.416195 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415929    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-textfile\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.416195 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.415998    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8281eeb4-bee7-4399-83c8-0e02a863793b-sys\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.416374 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.416216    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed8ce068-4b87-45cf-84aa-440fd4405a49-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.416443 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.416428    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8eef126c-9281-452b-b1f6-f81bd1fba198-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.417464 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.417434    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.417585 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.417541    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-tls\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.417905 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.417889    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.418013 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.417994    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.418475 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.418455    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8281eeb4-bee7-4399-83c8-0e02a863793b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.423217 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.423200    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkl84\" (UniqueName: \"kubernetes.io/projected/8eef126c-9281-452b-b1f6-f81bd1fba198-kube-api-access-dkl84\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.429132 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.429110    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltpxg\" (UniqueName: \"kubernetes.io/projected/8281eeb4-bee7-4399-83c8-0e02a863793b-kube-api-access-ltpxg\") pod \"node-exporter-q8q2g\" (UID: \"8281eeb4-bee7-4399-83c8-0e02a863793b\") " pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.432718 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.432696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jvj7\" (UniqueName: \"kubernetes.io/projected/ed8ce068-4b87-45cf-84aa-440fd4405a49-kube-api-access-2jvj7\") pod \"kube-state-metrics-56c5594674-q8g6l\" (UID: \"ed8ce068-4b87-45cf-84aa-440fd4405a49\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.459962 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.459941    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"
Jul 19 14:51:53.469189 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.469173    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q8q2g"
Jul 19 14:51:53.480381 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:53.480360    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8281eeb4_bee7_4399_83c8_0e02a863793b.slice/crio-10b3af8df72339abfa505b7490d75794aeedaa5159d8ba4e27aeeec9c346f2ed WatchSource:0}: Error finding container 10b3af8df72339abfa505b7490d75794aeedaa5159d8ba4e27aeeec9c346f2ed: Status 404 returned error can't find the container with id 10b3af8df72339abfa505b7490d75794aeedaa5159d8ba4e27aeeec9c346f2ed
Jul 19 14:51:53.482169 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.482148    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:51:53.586176 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.586153    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-q8g6l"]
Jul 19 14:51:53.588473 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:53.588445    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8ce068_4b87_45cf_84aa_440fd4405a49.slice/crio-2d8c4f4e2537b6b4070b792cd502912cb8f7f05d4fd8ede983ee112867d23ea4 WatchSource:0}: Error finding container 2d8c4f4e2537b6b4070b792cd502912cb8f7f05d4fd8ede983ee112867d23ea4: Status 404 returned error can't find the container with id 2d8c4f4e2537b6b4070b792cd502912cb8f7f05d4fd8ede983ee112867d23ea4
Jul 19 14:51:53.734029 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.733969    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-66c5bfdfc4-8bwrm"
Jul 19 14:51:53.803657 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.803623    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l" event={"ID":"ed8ce068-4b87-45cf-84aa-440fd4405a49","Type":"ContainerStarted","Data":"2d8c4f4e2537b6b4070b792cd502912cb8f7f05d4fd8ede983ee112867d23ea4"}
Jul 19 14:51:53.805176 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.805130    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8q2g" event={"ID":"8281eeb4-bee7-4399-83c8-0e02a863793b","Type":"ContainerStarted","Data":"10b3af8df72339abfa505b7490d75794aeedaa5159d8ba4e27aeeec9c346f2ed"}
Jul 19 14:51:53.920013 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:53.919978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:53.920172 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:53.920100    2572 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 19 14:51:53.920172 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:53.920159    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls podName:8eef126c-9281-452b-b1f6-f81bd1fba198 nodeName:}" failed. No retries permitted until 2026-07-19 14:51:54.92014431 +0000 UTC m=+316.621592342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-n7ngq" (UID: "8eef126c-9281-452b-b1f6-f81bd1fba198") : secret "openshift-state-metrics-tls" not found
Jul 19 14:51:54.810400 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:54.810325    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8q2g" event={"ID":"8281eeb4-bee7-4399-83c8-0e02a863793b","Type":"ContainerStarted","Data":"9c17025427910341ead326786ebcfe96521d3ef39437766023d14622dafb8cb8"}
Jul 19 14:51:54.927222 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:54.927191    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:54.929684 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:54.929660    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8eef126c-9281-452b-b1f6-f81bd1fba198-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-n7ngq\" (UID: \"8eef126c-9281-452b-b1f6-f81bd1fba198\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:54.936170 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:54.936149    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"
Jul 19 14:51:55.067555 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.067532    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq"]
Jul 19 14:51:55.071006 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:55.070964    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eef126c_9281_452b_b1f6_f81bd1fba198.slice/crio-03bdb408a45f76c84e6bfea382b8aede984257553a2b4a0a6030d2ca9372c3e4 WatchSource:0}: Error finding container 03bdb408a45f76c84e6bfea382b8aede984257553a2b4a0a6030d2ca9372c3e4: Status 404 returned error can't find the container with id 03bdb408a45f76c84e6bfea382b8aede984257553a2b4a0a6030d2ca9372c3e4
Jul 19 14:51:55.434397 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.434294    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc" podUID="0a4f6541-420b-4d0a-841f-4903afb44f58" containerName="registry" containerID="cri-o://dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800" gracePeriod=30
Jul 19 14:51:55.673536 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.673513    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:55.732849 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732769    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.732849 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732808    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-trusted-ca\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.732849 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732833    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-installation-pull-secrets\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.732849 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732850    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-image-registry-private-configuration\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.733164 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732875    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-certificates\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.733164 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732916    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-bound-sa-token\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.733164 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.732960    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a4f6541-420b-4d0a-841f-4903afb44f58-ca-trust-extracted\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.733402 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.733371    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:51:55.733582 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.733555    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 14:51:55.735527 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.735493    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:51:55.735625 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.735580    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 14:51:55.735692 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.735669    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:51:55.735759 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.735715    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:51:55.741946 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.741915    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4f6541-420b-4d0a-841f-4903afb44f58-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:51:55.816050 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.816000    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l" event={"ID":"ed8ce068-4b87-45cf-84aa-440fd4405a49","Type":"ContainerStarted","Data":"c454a4db3cc73ec64e57cfe808fdc4efd51cb8508b06ff18121eeab9b3c5bdff"}
Jul 19 14:51:55.816455 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.816082    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l" event={"ID":"ed8ce068-4b87-45cf-84aa-440fd4405a49","Type":"ContainerStarted","Data":"66bdbe44c7653424ecbd26dd642eb52731d7898f1fb882559d7a1577bac501e2"}
Jul 19 14:51:55.816455 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.816114    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l" event={"ID":"ed8ce068-4b87-45cf-84aa-440fd4405a49","Type":"ContainerStarted","Data":"d2db6382e002e48c10436c79f056ed65b04530125f412bc45eff4e625749112a"}
Jul 19 14:51:55.817498 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.817474    2572 generic.go:356] "Generic (PLEG): container finished" podID="8281eeb4-bee7-4399-83c8-0e02a863793b" containerID="9c17025427910341ead326786ebcfe96521d3ef39437766023d14622dafb8cb8" exitCode=0
Jul 19 14:51:55.817602 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.817546    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8q2g" event={"ID":"8281eeb4-bee7-4399-83c8-0e02a863793b","Type":"ContainerDied","Data":"9c17025427910341ead326786ebcfe96521d3ef39437766023d14622dafb8cb8"}
Jul 19 14:51:55.818794 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.818772    2572 generic.go:356] "Generic (PLEG): container finished" podID="0a4f6541-420b-4d0a-841f-4903afb44f58" containerID="dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800" exitCode=0
Jul 19 14:51:55.818880 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.818817    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc"
Jul 19 14:51:55.818880 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.818850    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc" event={"ID":"0a4f6541-420b-4d0a-841f-4903afb44f58","Type":"ContainerDied","Data":"dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800"}
Jul 19 14:51:55.819011 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.818890    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84f985cd78-z6vnc" event={"ID":"0a4f6541-420b-4d0a-841f-4903afb44f58","Type":"ContainerDied","Data":"fab81d535adaed429f5aba2ee4d656c552125a4749558a919f20dbcda6f03bd5"}
Jul 19 14:51:55.819011 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.818918    2572 scope.go:117] "RemoveContainer" containerID="dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800"
Jul 19 14:51:55.820744 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.820727    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq" event={"ID":"8eef126c-9281-452b-b1f6-f81bd1fba198","Type":"ContainerStarted","Data":"e2af37363e0052fa828939962d62a535a43360318970bbe9210524b3702e12f4"}
Jul 19 14:51:55.820842 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.820761    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq" event={"ID":"8eef126c-9281-452b-b1f6-f81bd1fba198","Type":"ContainerStarted","Data":"ab06bfeb95955c33fc6b2a8a3a959559ef4e9a795d6baa8e5ed6b59685943e92"}
Jul 19 14:51:55.820842 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.820780    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq" event={"ID":"8eef126c-9281-452b-b1f6-f81bd1fba198","Type":"ContainerStarted","Data":"03bdb408a45f76c84e6bfea382b8aede984257553a2b4a0a6030d2ca9372c3e4"}
Jul 19 14:51:55.828012 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.827995    2572 scope.go:117] "RemoveContainer" containerID="dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800"
Jul 19 14:51:55.828267 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:51:55.828246    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800\": container with ID starting with dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800 not found: ID does not exist" containerID="dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800"
Jul 19 14:51:55.828347 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.828279    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800"} err="failed to get container status \"dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800\": rpc error: code = NotFound desc = could not find container \"dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800\": container with ID starting with dfe7e7af28cef3a9756293ec592471c24d226d16a8c15aa0ff2c9f7892314800 not found: ID does not exist"
Jul 19 14:51:55.833608 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.833544    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-q8g6l" podStartSLOduration=1.4359271439999999 podStartE2EDuration="2.833531768s" totalImagesPullingTime="1.397604624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:55.832572686 +0000 UTC m=+317.534020741" watchObservedRunningTime="2026-07-19 14:51:55.833531768 +0000 UTC m=+317.534979800"
Jul 19 14:51:55.833817 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.833779    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bttvh\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-kube-api-access-bttvh\") pod \"0a4f6541-420b-4d0a-841f-4903afb44f58\" (UID: \"0a4f6541-420b-4d0a-841f-4903afb44f58\") "
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834267    2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-installation-pull-secrets\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834390    2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a4f6541-420b-4d0a-841f-4903afb44f58-image-registry-private-configuration\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834415    2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-certificates\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834477    2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-bound-sa-token\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834491    2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a4f6541-420b-4d0a-841f-4903afb44f58-ca-trust-extracted\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834504    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-registry-tls\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.834571 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.834517    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a4f6541-420b-4d0a-841f-4903afb44f58-trusted-ca\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:55.835983 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.835964    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-kube-api-access-bttvh" (OuterVolumeSpecName: "kube-api-access-bttvh") pod "0a4f6541-420b-4d0a-841f-4903afb44f58" (UID: "0a4f6541-420b-4d0a-841f-4903afb44f58"). InnerVolumeSpecName "kube-api-access-bttvh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:51:55.935666 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:55.935638    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bttvh\" (UniqueName: \"kubernetes.io/projected/0a4f6541-420b-4d0a-841f-4903afb44f58-kube-api-access-bttvh\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:51:56.143103 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.143042    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f985cd78-z6vnc"]
Jul 19 14:51:56.149937 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.149913    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84f985cd78-z6vnc"]
Jul 19 14:51:56.174688 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.174663    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7656454fbb-qwdr6"]
Jul 19 14:51:56.174971 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.174955    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4f6541-420b-4d0a-841f-4903afb44f58" containerName="registry"
Jul 19 14:51:56.175044 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.174974    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4f6541-420b-4d0a-841f-4903afb44f58" containerName="registry"
Jul 19 14:51:56.175044 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.175038    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4f6541-420b-4d0a-841f-4903afb44f58" containerName="registry"
Jul 19 14:51:56.178657 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.178634    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.181387 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181361    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 19 14:51:56.181517 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181367    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 19 14:51:56.181517 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181478    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ap5e5ncm191j8\""
Jul 19 14:51:56.181625 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181530    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2lpr5\""
Jul 19 14:51:56.181690 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181676    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 19 14:51:56.181737 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181728    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 19 14:51:56.181974 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.181729    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 19 14:51:56.193104 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.193071    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7656454fbb-qwdr6"]
Jul 19 14:51:56.239055 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-grpc-tls\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239200 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239085    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239200 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239111    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239200 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239142    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239200 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239185    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-metrics-client-ca\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239386 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239283    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-tls\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239386 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239359    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.239466 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.239393    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rm8j\" (UniqueName: \"kubernetes.io/projected/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-kube-api-access-6rm8j\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340415 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340374    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-grpc-tls\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340585 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340439    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340585 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340466    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340585 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340496    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340585 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-metrics-client-ca\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340585 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-tls\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340842 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340611    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.340842 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.340745    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rm8j\" (UniqueName: \"kubernetes.io/projected/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-kube-api-access-6rm8j\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.341576 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.341546    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-metrics-client-ca\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.343270 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.343244    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.343393 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.343372    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-grpc-tls\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.343625 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.343603    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.343738 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.343719    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-tls\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.343791 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.343768    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.343956 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.343940    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.351924 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.351893    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rm8j\" (UniqueName: \"kubernetes.io/projected/cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b-kube-api-access-6rm8j\") pod \"thanos-querier-7656454fbb-qwdr6\" (UID: \"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b\") " pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.489422 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.489387    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:51:56.619587 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.619556    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7656454fbb-qwdr6"]
Jul 19 14:51:56.622032 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:56.621999    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd3dfa90_bdd8_46b8_818c_deb07a6cdb2b.slice/crio-b63916f9a6e5e8dca8a73d1945e5530e3524313ee788e604f0085c78682fed3f WatchSource:0}: Error finding container b63916f9a6e5e8dca8a73d1945e5530e3524313ee788e604f0085c78682fed3f: Status 404 returned error can't find the container with id b63916f9a6e5e8dca8a73d1945e5530e3524313ee788e604f0085c78682fed3f
Jul 19 14:51:56.785612 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.785532    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a4f6541-420b-4d0a-841f-4903afb44f58" path="/var/lib/kubelet/pods/0a4f6541-420b-4d0a-841f-4903afb44f58/volumes"
Jul 19 14:51:56.826906 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.826875    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8q2g" event={"ID":"8281eeb4-bee7-4399-83c8-0e02a863793b","Type":"ContainerStarted","Data":"11de56d846ec19ece593231ac07f05b01a2f71849b1d146faa760d908a40b0fe"}
Jul 19 14:51:56.827294 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.826932    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q8q2g" event={"ID":"8281eeb4-bee7-4399-83c8-0e02a863793b","Type":"ContainerStarted","Data":"97218654c8ee5c4229e9b0da4ce2772625f9687ce6aedb6d75072b2f1b185f80"}
Jul 19 14:51:56.828022 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.828003    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"b63916f9a6e5e8dca8a73d1945e5530e3524313ee788e604f0085c78682fed3f"}
Jul 19 14:51:56.830785 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.830765    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq" event={"ID":"8eef126c-9281-452b-b1f6-f81bd1fba198","Type":"ContainerStarted","Data":"f3bdf1b579f417583da6f9a8c8c6f4aacce95b759bd428abad938e91422ec920"}
Jul 19 14:51:56.845197 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.845160    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q8q2g" podStartSLOduration=2.6705866069999997 podStartE2EDuration="3.845149156s" totalImagesPullingTime="1.094453562s" totalInitContainerRuntime="80.108987ms" isStatefulPod=true podCreationTimestamp="2026-07-19 14:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:56.843575923 +0000 UTC m=+318.545023978" watchObservedRunningTime="2026-07-19 14:51:56.845149156 +0000 UTC m=+318.546597227"
Jul 19 14:51:56.859816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:56.859777    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-n7ngq" podStartSLOduration=2.7461188 podStartE2EDuration="3.859762428s" totalImagesPullingTime="1.113643628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:56.859469631 +0000 UTC m=+318.560917693" watchObservedRunningTime="2026-07-19 14:51:56.859762428 +0000 UTC m=+318.561210483"
Jul 19 14:51:57.392558 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.392524    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79898cf4b8-csrkk"]
Jul 19 14:51:57.395813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.395792    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.400076 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.400046    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 19 14:51:57.400076 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.400065    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 19 14:51:57.400234 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.400049    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-edgkuu9ot6bf7\""
Jul 19 14:51:57.400234 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.400049    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 19 14:51:57.400369 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.400065    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 19 14:51:57.400369 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.400046    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-8vs8w\""
Jul 19 14:51:57.406095 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.406074    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79898cf4b8-csrkk"]
Jul 19 14:51:57.449125 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449097    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f8849838-b127-4594-8040-1e3345fef0e0-audit-log\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.449250 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449140    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-client-ca-bundle\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.449250 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449187    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-secret-metrics-server-client-certs\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.449250 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449212    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8849838-b127-4594-8040-1e3345fef0e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.449551 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449249    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm6ml\" (UniqueName: \"kubernetes.io/projected/f8849838-b127-4594-8040-1e3345fef0e0-kube-api-access-nm6ml\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.449551 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f8849838-b127-4594-8040-1e3345fef0e0-metrics-server-audit-profiles\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.449551 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.449320    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-secret-metrics-server-tls\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550105 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550067    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f8849838-b127-4594-8040-1e3345fef0e0-audit-log\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550120    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-client-ca-bundle\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550165    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-secret-metrics-server-client-certs\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550188    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8849838-b127-4594-8040-1e3345fef0e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550226    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm6ml\" (UniqueName: \"kubernetes.io/projected/f8849838-b127-4594-8040-1e3345fef0e0-kube-api-access-nm6ml\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550267    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f8849838-b127-4594-8040-1e3345fef0e0-metrics-server-audit-profiles\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550292    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-secret-metrics-server-tls\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.550622 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.550513    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f8849838-b127-4594-8040-1e3345fef0e0-audit-log\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.551096 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.551067    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f8849838-b127-4594-8040-1e3345fef0e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.551584 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.551549    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f8849838-b127-4594-8040-1e3345fef0e0-metrics-server-audit-profiles\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.553348 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.553302    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-secret-metrics-server-client-certs\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.553496 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.553473    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-secret-metrics-server-tls\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.553567 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.553509    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f8849838-b127-4594-8040-1e3345fef0e0-client-ca-bundle\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.560654 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.560618    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm6ml\" (UniqueName: \"kubernetes.io/projected/f8849838-b127-4594-8040-1e3345fef0e0-kube-api-access-nm6ml\") pod \"metrics-server-79898cf4b8-csrkk\" (UID: \"f8849838-b127-4594-8040-1e3345fef0e0\") " pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.705218 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.705136    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:51:57.853435 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.853403    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79898cf4b8-csrkk"]
Jul 19 14:51:57.856678 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:57.856656    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8849838_b127_4594_8040_1e3345fef0e0.slice/crio-75fb1652e772397c28a193b5aa7c1ddab302fdc8953c6130a36dd5076238ee76 WatchSource:0}: Error finding container 75fb1652e772397c28a193b5aa7c1ddab302fdc8953c6130a36dd5076238ee76: Status 404 returned error can't find the container with id 75fb1652e772397c28a193b5aa7c1ddab302fdc8953c6130a36dd5076238ee76
Jul 19 14:51:57.878237 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.878209    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"]
Jul 19 14:51:57.883505 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.883003    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:57.886036 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.886006    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 19 14:51:57.886036 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.886021    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6b5gs\""
Jul 19 14:51:57.892301 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.892186    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"]
Jul 19 14:51:57.954952 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.954921    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1f6f9ba-ec81-478d-83ae-eee4536efd47-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8xtnp\" (UID: \"c1f6f9ba-ec81-478d-83ae-eee4536efd47\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:57.955138 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:57.954971    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnfkn\" (UniqueName: \"kubernetes.io/projected/c1f6f9ba-ec81-478d-83ae-eee4536efd47-kube-api-access-tnfkn\") pod \"monitoring-plugin-7b7b7bc8bf-8xtnp\" (UID: \"c1f6f9ba-ec81-478d-83ae-eee4536efd47\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:58.056067 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.055988    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1f6f9ba-ec81-478d-83ae-eee4536efd47-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8xtnp\" (UID: \"c1f6f9ba-ec81-478d-83ae-eee4536efd47\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:58.056067 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.056044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnfkn\" (UniqueName: \"kubernetes.io/projected/c1f6f9ba-ec81-478d-83ae-eee4536efd47-kube-api-access-tnfkn\") pod \"monitoring-plugin-7b7b7bc8bf-8xtnp\" (UID: \"c1f6f9ba-ec81-478d-83ae-eee4536efd47\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:58.058232 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.058213    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c1f6f9ba-ec81-478d-83ae-eee4536efd47-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-8xtnp\" (UID: \"c1f6f9ba-ec81-478d-83ae-eee4536efd47\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:58.064001 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.063982    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnfkn\" (UniqueName: \"kubernetes.io/projected/c1f6f9ba-ec81-478d-83ae-eee4536efd47-kube-api-access-tnfkn\") pod \"monitoring-plugin-7b7b7bc8bf-8xtnp\" (UID: \"c1f6f9ba-ec81-478d-83ae-eee4536efd47\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:58.195871 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.195834    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:51:58.344330 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.344261    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"]
Jul 19 14:51:58.840181 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:58.840112    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk" event={"ID":"f8849838-b127-4594-8040-1e3345fef0e0","Type":"ContainerStarted","Data":"75fb1652e772397c28a193b5aa7c1ddab302fdc8953c6130a36dd5076238ee76"}
Jul 19 14:51:58.861867 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:51:58.861823    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f6f9ba_ec81_478d_83ae_eee4536efd47.slice/crio-671c7272e828a14b5ce08326724e4d04dcca28b41f5e077d6f5ef2f6f856c9dd WatchSource:0}: Error finding container 671c7272e828a14b5ce08326724e4d04dcca28b41f5e077d6f5ef2f6f856c9dd: Status 404 returned error can't find the container with id 671c7272e828a14b5ce08326724e4d04dcca28b41f5e077d6f5ef2f6f856c9dd
Jul 19 14:51:59.847736 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:59.847699    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"d96e2d8b14852d3e64885dc3400f5f134a45a471c361724830b1df3dd57c5e6b"}
Jul 19 14:51:59.847892 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:59.847812    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"66b0a796266d5829dda38dad625d45bd6eebac4c319a1a7e6f97057463e135bf"}
Jul 19 14:51:59.847892 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:59.847852    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"d7867c5369dea89ce39f9a9819d02f3eb4aa8feb8ded650ac633137ba1f1ca4b"}
Jul 19 14:51:59.849541 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:59.849517    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk" event={"ID":"f8849838-b127-4594-8040-1e3345fef0e0","Type":"ContainerStarted","Data":"c85fe5c5c991094cd092060bbe3337aa35f124d23cd1e41a7eab7121826b0a71"}
Jul 19 14:51:59.850799 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:59.850767    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp" event={"ID":"c1f6f9ba-ec81-478d-83ae-eee4536efd47","Type":"ContainerStarted","Data":"671c7272e828a14b5ce08326724e4d04dcca28b41f5e077d6f5ef2f6f856c9dd"}
Jul 19 14:51:59.865836 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:51:59.865782    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk" podStartSLOduration=1.258960812 podStartE2EDuration="2.865766173s" totalImagesPullingTime="1.606805361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:51:59.865694179 +0000 UTC m=+321.567142233" watchObservedRunningTime="2026-07-19 14:51:59.865766173 +0000 UTC m=+321.567214229"
Jul 19 14:52:01.860282 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.860244    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"442be8711e3e842e6f33e2564e5f71b21b1785ab5bdebe64b6967e26d1c159d2"}
Jul 19 14:52:01.860745 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.860314    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"7e312d9263dff3ab963213c6dd49d0907c9875f124c81739421d1f9ee698d5dd"}
Jul 19 14:52:01.860745 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.860367    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" event={"ID":"cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b","Type":"ContainerStarted","Data":"ff4415a08515552a9bb5c34ad13f1134d553a899da6e6e8280824c57d2a06227"}
Jul 19 14:52:01.860745 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.860414    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:52:01.861657 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.861633    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp" event={"ID":"c1f6f9ba-ec81-478d-83ae-eee4536efd47","Type":"ContainerStarted","Data":"83cc12163dd9545f9fb56ca91427983895fdca234afce782f8e7c5740c218ee4"}
Jul 19 14:52:01.861824 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.861814    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:52:01.866630 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.866613    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp"
Jul 19 14:52:01.883347 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.883292    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6" podStartSLOduration=2.022116762 podStartE2EDuration="5.883277149s" totalImagesPullingTime="3.861160387s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:52:01.881380609 +0000 UTC m=+323.582828662" watchObservedRunningTime="2026-07-19 14:52:01.883277149 +0000 UTC m=+323.584725202"
Jul 19 14:52:01.896300 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:01.896266    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-8xtnp" podStartSLOduration=2.705732099 podStartE2EDuration="4.896253468s" totalImagesPullingTime="2.190521369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:52:01.895617305 +0000 UTC m=+323.597065358" watchObservedRunningTime="2026-07-19 14:52:01.896253468 +0000 UTC m=+323.597701521"
Jul 19 14:52:07.872221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:07.872195    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7656454fbb-qwdr6"
Jul 19 14:52:10.903611 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:10.903573    2572 generic.go:356] "Generic (PLEG): container finished" podID="bec9c880-4f12-4177-aac1-f6474b24054c" containerID="ef5aa63c841b40a3d8792ef1a04269ed9158c4890bec90627b0c41f8878fab85" exitCode=0
Jul 19 14:52:10.903985 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:10.903645    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd" event={"ID":"bec9c880-4f12-4177-aac1-f6474b24054c","Type":"ContainerDied","Data":"ef5aa63c841b40a3d8792ef1a04269ed9158c4890bec90627b0c41f8878fab85"}
Jul 19 14:52:10.903985 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:10.903942    2572 scope.go:117] "RemoveContainer" containerID="ef5aa63c841b40a3d8792ef1a04269ed9158c4890bec90627b0c41f8878fab85"
Jul 19 14:52:11.909785 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:11.909755    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-ls8bd" event={"ID":"bec9c880-4f12-4177-aac1-f6474b24054c","Type":"ContainerStarted","Data":"aa099395249faaffa92c1fda45e153215d38606534c6cbc4915c136467659d57"}
Jul 19 14:52:17.705529 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:17.705486    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:52:17.705907 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:17.705552    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:52:37.711265 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:37.711187    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:52:37.714964 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:37.714944    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79898cf4b8-csrkk"
Jul 19 14:52:52.720478 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:52.720421    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:52:52.722832 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:52.722813    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36ff5be0-b9b5-46b8-a4d3-02765a4a6331-metrics-certs\") pod \"network-metrics-daemon-rkxt7\" (UID: \"36ff5be0-b9b5-46b8-a4d3-02765a4a6331\") " pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:52:52.784363 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:52.783285    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5p5rq\""
Jul 19 14:52:52.790532 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:52.790513    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkxt7"
Jul 19 14:52:52.919634 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:52.919604    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkxt7"]
Jul 19 14:52:52.923656 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:52:52.923628    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ff5be0_b9b5_46b8_a4d3_02765a4a6331.slice/crio-c9aac4f93afbc49e740b8eeb7ae9822015b9a98ed778fef51450dee9c63541ba WatchSource:0}: Error finding container c9aac4f93afbc49e740b8eeb7ae9822015b9a98ed778fef51450dee9c63541ba: Status 404 returned error can't find the container with id c9aac4f93afbc49e740b8eeb7ae9822015b9a98ed778fef51450dee9c63541ba
Jul 19 14:52:53.062911 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:53.062844    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkxt7" event={"ID":"36ff5be0-b9b5-46b8-a4d3-02765a4a6331","Type":"ContainerStarted","Data":"c9aac4f93afbc49e740b8eeb7ae9822015b9a98ed778fef51450dee9c63541ba"}
Jul 19 14:52:54.069911 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:54.069874    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkxt7" event={"ID":"36ff5be0-b9b5-46b8-a4d3-02765a4a6331","Type":"ContainerStarted","Data":"a8a15a2246019e4034c095f05af8cf2e5bd7932b09430a53e099b32bc4e8a548"}
Jul 19 14:52:54.070282 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:54.069926    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkxt7" event={"ID":"36ff5be0-b9b5-46b8-a4d3-02765a4a6331","Type":"ContainerStarted","Data":"c0a5a01f050188d71d12d722ba7a616e08e41bd76cbb61c42b3d3a1dc6bda3cb"}
Jul 19 14:52:54.086773 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:52:54.086723    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rkxt7" podStartSLOduration=374.118726937 podStartE2EDuration="6m15.086706071s" totalImagesPullingTime="967.979134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:46:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:52:54.085644691 +0000 UTC m=+375.787092745" watchObservedRunningTime="2026-07-19 14:52:54.086706071 +0000 UTC m=+375.788154124"
Jul 19 14:55:17.822035 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.822001    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"]
Jul 19 14:55:17.825289 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.825271    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:17.827930 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.827909    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 19 14:55:17.828959 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.828944    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hkshb\""
Jul 19 14:55:17.829040 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.828994    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 19 14:55:17.834693 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.834672    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"]
Jul 19 14:55:17.944021 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.943988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:17.944183 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.944030    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vz82\" (UniqueName: \"kubernetes.io/projected/a69d3b13-2c10-4f52-a84b-c620fa81c42e-kube-api-access-4vz82\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:17.944183 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:17.944132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.044500 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.044469    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.044663 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.044510    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vz82\" (UniqueName: \"kubernetes.io/projected/a69d3b13-2c10-4f52-a84b-c620fa81c42e-kube-api-access-4vz82\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.044663 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.044571    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.044813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.044795    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.044899 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.044882    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.052664 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.052634    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vz82\" (UniqueName: \"kubernetes.io/projected/a69d3b13-2c10-4f52-a84b-c620fa81c42e-kube-api-access-4vz82\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.134795 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.134768    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:18.258878 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.258821    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"]
Jul 19 14:55:18.261001 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:55:18.260974    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda69d3b13_2c10_4f52_a84b_c620fa81c42e.slice/crio-1d9444e2771ca50d3cd932d293ac0910a3396e02ad7b6a2db45e97dd485ea358 WatchSource:0}: Error finding container 1d9444e2771ca50d3cd932d293ac0910a3396e02ad7b6a2db45e97dd485ea358: Status 404 returned error can't find the container with id 1d9444e2771ca50d3cd932d293ac0910a3396e02ad7b6a2db45e97dd485ea358
Jul 19 14:55:18.582482 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:18.582452    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn" event={"ID":"a69d3b13-2c10-4f52-a84b-c620fa81c42e","Type":"ContainerStarted","Data":"1d9444e2771ca50d3cd932d293ac0910a3396e02ad7b6a2db45e97dd485ea358"}
Jul 19 14:55:24.607577 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:24.607545    2572 generic.go:356] "Generic (PLEG): container finished" podID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerID="6fa7e8bd1f949e3cfee8b6dfe53915e87e57b571af77658f0c2c351c9acb8677" exitCode=0
Jul 19 14:55:24.607952 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:24.607622    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn" event={"ID":"a69d3b13-2c10-4f52-a84b-c620fa81c42e","Type":"ContainerDied","Data":"6fa7e8bd1f949e3cfee8b6dfe53915e87e57b571af77658f0c2c351c9acb8677"}
Jul 19 14:55:27.621651 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:27.621615    2572 generic.go:356] "Generic (PLEG): container finished" podID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerID="2b1c1c453da778449cef9ada2f54e579067c119824967638988dbe3adaf7676c" exitCode=0
Jul 19 14:55:27.622023 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:27.621701    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn" event={"ID":"a69d3b13-2c10-4f52-a84b-c620fa81c42e","Type":"ContainerDied","Data":"2b1c1c453da778449cef9ada2f54e579067c119824967638988dbe3adaf7676c"}
Jul 19 14:55:34.650668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:34.650640    2572 generic.go:356] "Generic (PLEG): container finished" podID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerID="73dab2cc856421244c6ac3b8c0a7049ef040aa5174034dd9695ca4509d1a3d3c" exitCode=0
Jul 19 14:55:34.650951 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:34.650721    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn" event={"ID":"a69d3b13-2c10-4f52-a84b-c620fa81c42e","Type":"ContainerDied","Data":"73dab2cc856421244c6ac3b8c0a7049ef040aa5174034dd9695ca4509d1a3d3c"}
Jul 19 14:55:35.775535 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.775511    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:35.893557 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.893528    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vz82\" (UniqueName: \"kubernetes.io/projected/a69d3b13-2c10-4f52-a84b-c620fa81c42e-kube-api-access-4vz82\") pod \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") "
Jul 19 14:55:35.893704 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.893579    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-util\") pod \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") "
Jul 19 14:55:35.893704 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.893610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-bundle\") pod \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\" (UID: \"a69d3b13-2c10-4f52-a84b-c620fa81c42e\") "
Jul 19 14:55:35.894380 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.894260    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-bundle" (OuterVolumeSpecName: "bundle") pod "a69d3b13-2c10-4f52-a84b-c620fa81c42e" (UID: "a69d3b13-2c10-4f52-a84b-c620fa81c42e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:55:35.895869 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.895842    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a69d3b13-2c10-4f52-a84b-c620fa81c42e-kube-api-access-4vz82" (OuterVolumeSpecName: "kube-api-access-4vz82") pod "a69d3b13-2c10-4f52-a84b-c620fa81c42e" (UID: "a69d3b13-2c10-4f52-a84b-c620fa81c42e"). InnerVolumeSpecName "kube-api-access-4vz82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:35.898190 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.898172    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-util" (OuterVolumeSpecName: "util") pod "a69d3b13-2c10-4f52-a84b-c620fa81c42e" (UID: "a69d3b13-2c10-4f52-a84b-c620fa81c42e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:55:35.994818 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.994754    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:55:35.994818 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.994778    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vz82\" (UniqueName: \"kubernetes.io/projected/a69d3b13-2c10-4f52-a84b-c620fa81c42e-kube-api-access-4vz82\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:55:35.994818 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:35.994786    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a69d3b13-2c10-4f52-a84b-c620fa81c42e-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:55:36.659273 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:36.659240    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn" event={"ID":"a69d3b13-2c10-4f52-a84b-c620fa81c42e","Type":"ContainerDied","Data":"1d9444e2771ca50d3cd932d293ac0910a3396e02ad7b6a2db45e97dd485ea358"}
Jul 19 14:55:36.659451 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:36.659280    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540q5csn"
Jul 19 14:55:36.659451 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:36.659293    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d9444e2771ca50d3cd932d293ac0910a3396e02ad7b6a2db45e97dd485ea358"
Jul 19 14:55:40.535824 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.535786    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"]
Jul 19 14:55:40.536199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536136    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="extract"
Jul 19 14:55:40.536199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536149    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="extract"
Jul 19 14:55:40.536199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536157    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="pull"
Jul 19 14:55:40.536199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536162    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="pull"
Jul 19 14:55:40.536199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536172    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="util"
Jul 19 14:55:40.536199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536178    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="util"
Jul 19 14:55:40.536394 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.536222    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a69d3b13-2c10-4f52-a84b-c620fa81c42e" containerName="extract"
Jul 19 14:55:40.573220 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.573192    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"]
Jul 19 14:55:40.573394 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.573303    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.578234 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.578209    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 19 14:55:40.578430 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.578285    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-t4567\""
Jul 19 14:55:40.578729 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.578714    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:55:40.634256 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.634215    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvwjc\" (UniqueName: \"kubernetes.io/projected/eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021-kube-api-access-nvwjc\") pod \"cert-manager-operator-controller-manager-55c444bf86-mbxrc\" (UID: \"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.634442 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.634285    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mbxrc\" (UID: \"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.735071 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.735027    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvwjc\" (UniqueName: \"kubernetes.io/projected/eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021-kube-api-access-nvwjc\") pod \"cert-manager-operator-controller-manager-55c444bf86-mbxrc\" (UID: \"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.735268 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.735093    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mbxrc\" (UID: \"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.735456 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.735439    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mbxrc\" (UID: \"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.743105 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.743084    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvwjc\" (UniqueName: \"kubernetes.io/projected/eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021-kube-api-access-nvwjc\") pod \"cert-manager-operator-controller-manager-55c444bf86-mbxrc\" (UID: \"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:40.882410 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:40.882380    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"
Jul 19 14:55:41.026391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:41.026321    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc"]
Jul 19 14:55:41.028778 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:55:41.028755    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7c86e4_dd0e_49f3_a7db_0e7b0d7ac021.slice/crio-f52ff343763e6b1d80a4f1a178097e84f12fdb8ad0135cf12a3991ca095139fb WatchSource:0}: Error finding container f52ff343763e6b1d80a4f1a178097e84f12fdb8ad0135cf12a3991ca095139fb: Status 404 returned error can't find the container with id f52ff343763e6b1d80a4f1a178097e84f12fdb8ad0135cf12a3991ca095139fb
Jul 19 14:55:41.679774 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:41.679745    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc" event={"ID":"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021","Type":"ContainerStarted","Data":"f52ff343763e6b1d80a4f1a178097e84f12fdb8ad0135cf12a3991ca095139fb"}
Jul 19 14:55:43.689415 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:43.689377    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc" event={"ID":"eb7c86e4-dd0e-49f3-a7db-0e7b0d7ac021","Type":"ContainerStarted","Data":"2f0a68233e44fb2cf043e46c69c167dc501f4c534b2acd4a5ff3518b4cc15998"}
Jul 19 14:55:43.713561 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:43.713494    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mbxrc" podStartSLOduration=1.671309879 podStartE2EDuration="3.713473231s" totalImagesPullingTime="2.042163352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:55:43.711458397 +0000 UTC m=+545.412906450" watchObservedRunningTime="2026-07-19 14:55:43.713473231 +0000 UTC m=+545.414921285"
Jul 19 14:55:45.014828 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.014794    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"]
Jul 19 14:55:45.034237 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.034201    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"]
Jul 19 14:55:45.034437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.034302    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.038541 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.038519    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hkshb\""
Jul 19 14:55:45.038541 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.038519    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 19 14:55:45.039609 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.039588    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 19 14:55:45.177307 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.177263    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dw4j\" (UniqueName: \"kubernetes.io/projected/df95f751-3de8-427a-a217-5d7ca373f8b1-kube-api-access-9dw4j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.177514 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.177323    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.177514 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.177433    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.278737 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.278652    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dw4j\" (UniqueName: \"kubernetes.io/projected/df95f751-3de8-427a-a217-5d7ca373f8b1-kube-api-access-9dw4j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.278737 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.278701    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.278922 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.278754    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.279107 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.279088    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.279143 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.279134    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.286789 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.286759    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dw4j\" (UniqueName: \"kubernetes.io/projected/df95f751-3de8-427a-a217-5d7ca373f8b1-kube-api-access-9dw4j\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.343802 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.343761    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:45.498124 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.498009    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"]
Jul 19 14:55:45.500308 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:55:45.500277    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf95f751_3de8_427a_a217_5d7ca373f8b1.slice/crio-f0aeb3bf23880d3ed8547fb5a62879906886fa41c848d2041b0946b2609be8c9 WatchSource:0}: Error finding container f0aeb3bf23880d3ed8547fb5a62879906886fa41c848d2041b0946b2609be8c9: Status 404 returned error can't find the container with id f0aeb3bf23880d3ed8547fb5a62879906886fa41c848d2041b0946b2609be8c9
Jul 19 14:55:45.699176 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.699141    2572 generic.go:356] "Generic (PLEG): container finished" podID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerID="ca6a3260feb84e6318219b6975b9e191236da01b51e9bc75f7d43162b89007be" exitCode=0
Jul 19 14:55:45.699379 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.699212    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx" event={"ID":"df95f751-3de8-427a-a217-5d7ca373f8b1","Type":"ContainerDied","Data":"ca6a3260feb84e6318219b6975b9e191236da01b51e9bc75f7d43162b89007be"}
Jul 19 14:55:45.699379 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:45.699262    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx" event={"ID":"df95f751-3de8-427a-a217-5d7ca373f8b1","Type":"ContainerStarted","Data":"f0aeb3bf23880d3ed8547fb5a62879906886fa41c848d2041b0946b2609be8c9"}
Jul 19 14:55:46.396917 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.396882    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"]
Jul 19 14:55:46.414234 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.414200    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"]
Jul 19 14:55:46.414424 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.414297    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.417017 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.416970    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 19 14:55:46.417017 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.417011    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qwlv2\""
Jul 19 14:55:46.418143 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.418123    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 19 14:55:46.489702 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.489664    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dg5hx\" (UID: \"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.489702 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.489705    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhnc5\" (UniqueName: \"kubernetes.io/projected/f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e-kube-api-access-qhnc5\") pod \"cert-manager-cainjector-559cdd8c87-dg5hx\" (UID: \"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.590813 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.590772    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dg5hx\" (UID: \"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.591003 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.590817    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhnc5\" (UniqueName: \"kubernetes.io/projected/f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e-kube-api-access-qhnc5\") pod \"cert-manager-cainjector-559cdd8c87-dg5hx\" (UID: \"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.599630 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.599601    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dg5hx\" (UID: \"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.599776 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.599663    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhnc5\" (UniqueName: \"kubernetes.io/projected/f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e-kube-api-access-qhnc5\") pod \"cert-manager-cainjector-559cdd8c87-dg5hx\" (UID: \"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.734201 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.734172    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"
Jul 19 14:55:46.902933 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:46.902907    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx"]
Jul 19 14:55:46.905744 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:55:46.905700    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1a0a125_b5bd_42f5_8c2d_48ecaf13fc1e.slice/crio-91b93033336f1d199cfc08e2afa0ded52440eea87c929b0164c6de9f4116241c WatchSource:0}: Error finding container 91b93033336f1d199cfc08e2afa0ded52440eea87c929b0164c6de9f4116241c: Status 404 returned error can't find the container with id 91b93033336f1d199cfc08e2afa0ded52440eea87c929b0164c6de9f4116241c
Jul 19 14:55:47.712970 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:47.712932    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx" event={"ID":"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e","Type":"ContainerStarted","Data":"91b93033336f1d199cfc08e2afa0ded52440eea87c929b0164c6de9f4116241c"}
Jul 19 14:55:48.720273 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:48.720235    2572 generic.go:356] "Generic (PLEG): container finished" podID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerID="f8104319ecd0d02161c6c1b5200d5983a43007356f32f78965662e8415d1d83f" exitCode=0
Jul 19 14:55:48.720772 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:48.720354    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx" event={"ID":"df95f751-3de8-427a-a217-5d7ca373f8b1","Type":"ContainerDied","Data":"f8104319ecd0d02161c6c1b5200d5983a43007356f32f78965662e8415d1d83f"}
Jul 19 14:55:49.726655 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:49.726618    2572 generic.go:356] "Generic (PLEG): container finished" podID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerID="dad52ab36f434eb9bc32d59d26e05bc7cb181e77ce9896580cccd6310116cea1" exitCode=0
Jul 19 14:55:49.727014 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:49.726709    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx" event={"ID":"df95f751-3de8-427a-a217-5d7ca373f8b1","Type":"ContainerDied","Data":"dad52ab36f434eb9bc32d59d26e05bc7cb181e77ce9896580cccd6310116cea1"}
Jul 19 14:55:50.732948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:50.732909    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx" event={"ID":"f1a0a125-b5bd-42f5-8c2d-48ecaf13fc1e","Type":"ContainerStarted","Data":"3139f201906c0a86331f25f5f3582156351ad7ac51f6aa51480e28068455d85a"}
Jul 19 14:55:50.749172 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:50.749106    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dg5hx" podStartSLOduration=1.95804131 podStartE2EDuration="4.749084422s" totalImagesPullingTime="2.791043112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:55:50.747664098 +0000 UTC m=+552.449112150" watchObservedRunningTime="2026-07-19 14:55:50.749084422 +0000 UTC m=+552.450532477"
Jul 19 14:55:50.859297 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:50.859275    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:51.030660 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.030583    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-bundle\") pod \"df95f751-3de8-427a-a217-5d7ca373f8b1\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") "
Jul 19 14:55:51.030660 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.030641    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-util\") pod \"df95f751-3de8-427a-a217-5d7ca373f8b1\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") "
Jul 19 14:55:51.030856 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.030667    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dw4j\" (UniqueName: \"kubernetes.io/projected/df95f751-3de8-427a-a217-5d7ca373f8b1-kube-api-access-9dw4j\") pod \"df95f751-3de8-427a-a217-5d7ca373f8b1\" (UID: \"df95f751-3de8-427a-a217-5d7ca373f8b1\") "
Jul 19 14:55:51.031052 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.031033    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-bundle" (OuterVolumeSpecName: "bundle") pod "df95f751-3de8-427a-a217-5d7ca373f8b1" (UID: "df95f751-3de8-427a-a217-5d7ca373f8b1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:55:51.033064 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.033037    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df95f751-3de8-427a-a217-5d7ca373f8b1-kube-api-access-9dw4j" (OuterVolumeSpecName: "kube-api-access-9dw4j") pod "df95f751-3de8-427a-a217-5d7ca373f8b1" (UID: "df95f751-3de8-427a-a217-5d7ca373f8b1"). InnerVolumeSpecName "kube-api-access-9dw4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:55:51.035962 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.035917    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-util" (OuterVolumeSpecName: "util") pod "df95f751-3de8-427a-a217-5d7ca373f8b1" (UID: "df95f751-3de8-427a-a217-5d7ca373f8b1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:55:51.131444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.131410    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:55:51.131444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.131436    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df95f751-3de8-427a-a217-5d7ca373f8b1-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:55:51.131444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.131444    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dw4j\" (UniqueName: \"kubernetes.io/projected/df95f751-3de8-427a-a217-5d7ca373f8b1-kube-api-access-9dw4j\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:55:51.739369 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.739323    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx"
Jul 19 14:55:51.739831 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.739365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq9jmx" event={"ID":"df95f751-3de8-427a-a217-5d7ca373f8b1","Type":"ContainerDied","Data":"f0aeb3bf23880d3ed8547fb5a62879906886fa41c848d2041b0946b2609be8c9"}
Jul 19 14:55:51.739831 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:55:51.739451    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0aeb3bf23880d3ed8547fb5a62879906886fa41c848d2041b0946b2609be8c9"
Jul 19 14:56:05.262277 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262240    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"]
Jul 19 14:56:05.262668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262577    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="util"
Jul 19 14:56:05.262668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262589    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="util"
Jul 19 14:56:05.262668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262600    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="extract"
Jul 19 14:56:05.262668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262608    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="extract"
Jul 19 14:56:05.262668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262619    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="pull"
Jul 19 14:56:05.262668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262625    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="pull"
Jul 19 14:56:05.262862 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.262676    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="df95f751-3de8-427a-a217-5d7ca373f8b1" containerName="extract"
Jul 19 14:56:05.265668 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.265650    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.268227 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.268207    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 19 14:56:05.269250 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.269233    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-hkshb\""
Jul 19 14:56:05.269374 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.269276    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 19 14:56:05.275912 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.275885    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"]
Jul 19 14:56:05.355134 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.355096    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.355134 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.355141    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.355391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.355174    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjqhc\" (UniqueName: \"kubernetes.io/projected/cb203602-f634-4cbc-95c4-77cfedbb0284-kube-api-access-jjqhc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.456426 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.456373    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.456426 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.456419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.456712 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.456454    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjqhc\" (UniqueName: \"kubernetes.io/projected/cb203602-f634-4cbc-95c4-77cfedbb0284-kube-api-access-jjqhc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.456895 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.456869    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.456895 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.456869    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.465556 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.465524    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjqhc\" (UniqueName: \"kubernetes.io/projected/cb203602-f634-4cbc-95c4-77cfedbb0284-kube-api-access-jjqhc\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.575722 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.575572    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:05.713966 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.713934    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"]
Jul 19 14:56:05.716905 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:56:05.716882    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb203602_f634_4cbc_95c4_77cfedbb0284.slice/crio-cd49109ef8b7940c84f06d641c9bdc67a5a0aba92226745d0cd1e71967c58af0 WatchSource:0}: Error finding container cd49109ef8b7940c84f06d641c9bdc67a5a0aba92226745d0cd1e71967c58af0: Status 404 returned error can't find the container with id cd49109ef8b7940c84f06d641c9bdc67a5a0aba92226745d0cd1e71967c58af0
Jul 19 14:56:05.808253 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.808219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97" event={"ID":"cb203602-f634-4cbc-95c4-77cfedbb0284","Type":"ContainerStarted","Data":"0c13f9c081c1531494adc7890d902d21c31d352031e53427944a839176be38ee"}
Jul 19 14:56:05.808426 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:05.808266    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97" event={"ID":"cb203602-f634-4cbc-95c4-77cfedbb0284","Type":"ContainerStarted","Data":"cd49109ef8b7940c84f06d641c9bdc67a5a0aba92226745d0cd1e71967c58af0"}
Jul 19 14:56:06.816941 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:06.816856    2572 generic.go:356] "Generic (PLEG): container finished" podID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerID="0c13f9c081c1531494adc7890d902d21c31d352031e53427944a839176be38ee" exitCode=0
Jul 19 14:56:06.817300 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:06.816945    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97" event={"ID":"cb203602-f634-4cbc-95c4-77cfedbb0284","Type":"ContainerDied","Data":"0c13f9c081c1531494adc7890d902d21c31d352031e53427944a839176be38ee"}
Jul 19 14:56:07.823053 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:07.823016    2572 generic.go:356] "Generic (PLEG): container finished" podID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerID="c5478090b172f4ca86d921357620fe926fadaa2c163a61ffabbd92e30b007a44" exitCode=0
Jul 19 14:56:07.823520 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:07.823110    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97" event={"ID":"cb203602-f634-4cbc-95c4-77cfedbb0284","Type":"ContainerDied","Data":"c5478090b172f4ca86d921357620fe926fadaa2c163a61ffabbd92e30b007a44"}
Jul 19 14:56:08.829731 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:08.829689    2572 generic.go:356] "Generic (PLEG): container finished" podID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerID="deada6645626618028bc601591e44ff99bda5f2bdde7bdb6fa2dbec340ab3b00" exitCode=0
Jul 19 14:56:08.830148 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:08.829763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97" event={"ID":"cb203602-f634-4cbc-95c4-77cfedbb0284","Type":"ContainerDied","Data":"deada6645626618028bc601591e44ff99bda5f2bdde7bdb6fa2dbec340ab3b00"}
Jul 19 14:56:09.980070 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.980039    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:09.987902 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.987871    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-util\") pod \"cb203602-f634-4cbc-95c4-77cfedbb0284\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") "
Jul 19 14:56:09.988063 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.987919    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-bundle\") pod \"cb203602-f634-4cbc-95c4-77cfedbb0284\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") "
Jul 19 14:56:09.988063 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.987990    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjqhc\" (UniqueName: \"kubernetes.io/projected/cb203602-f634-4cbc-95c4-77cfedbb0284-kube-api-access-jjqhc\") pod \"cb203602-f634-4cbc-95c4-77cfedbb0284\" (UID: \"cb203602-f634-4cbc-95c4-77cfedbb0284\") "
Jul 19 14:56:09.988709 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.988682    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-bundle" (OuterVolumeSpecName: "bundle") pod "cb203602-f634-4cbc-95c4-77cfedbb0284" (UID: "cb203602-f634-4cbc-95c4-77cfedbb0284"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:56:09.990330 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.990298    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb203602-f634-4cbc-95c4-77cfedbb0284-kube-api-access-jjqhc" (OuterVolumeSpecName: "kube-api-access-jjqhc") pod "cb203602-f634-4cbc-95c4-77cfedbb0284" (UID: "cb203602-f634-4cbc-95c4-77cfedbb0284"). InnerVolumeSpecName "kube-api-access-jjqhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:56:09.993461 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:09.993432    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-util" (OuterVolumeSpecName: "util") pod "cb203602-f634-4cbc-95c4-77cfedbb0284" (UID: "cb203602-f634-4cbc-95c4-77cfedbb0284"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:56:10.088536 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:10.088433    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:56:10.088536 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:10.088460    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb203602-f634-4cbc-95c4-77cfedbb0284-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:56:10.088536 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:10.088469    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjqhc\" (UniqueName: \"kubernetes.io/projected/cb203602-f634-4cbc-95c4-77cfedbb0284-kube-api-access-jjqhc\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:56:10.841526 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:10.841480    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97" event={"ID":"cb203602-f634-4cbc-95c4-77cfedbb0284","Type":"ContainerDied","Data":"cd49109ef8b7940c84f06d641c9bdc67a5a0aba92226745d0cd1e71967c58af0"}
Jul 19 14:56:10.841526 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:10.841515    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2cs97"
Jul 19 14:56:10.841729 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:10.841544    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd49109ef8b7940c84f06d641c9bdc67a5a0aba92226745d0cd1e71967c58af0"
Jul 19 14:56:17.953122 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953086    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"]
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953549    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="util"
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953569    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="util"
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953585    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="extract"
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953594    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="extract"
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953622    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="pull"
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953630    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="pull"
Jul 19 14:56:17.953875 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.953829    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb203602-f634-4cbc-95c4-77cfedbb0284" containerName="extract"
Jul 19 14:56:17.958439 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.958418    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:17.961211 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.961185    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 19 14:56:17.961369 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.961226    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 19 14:56:17.961369 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.961359    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 19 14:56:17.961535 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.961512    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 19 14:56:17.962003 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.961990    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2qtz6\""
Jul 19 14:56:17.971972 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:17.971945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"]
Jul 19 14:56:18.058684 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.058642    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prph6\" (UniqueName: \"kubernetes.io/projected/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-kube-api-access-prph6\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.058873 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.058718    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.058873 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.058754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-manifests\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.058873 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.058783    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.058873 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.058800    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-tmp\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.160109 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.160004    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.160109 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.160124    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-tmp\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.160381 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.160146    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prph6\" (UniqueName: \"kubernetes.io/projected/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-kube-api-access-prph6\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.160381 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.160233    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.160381 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.160271    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-manifests\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.160658 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.160639    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-manifests\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.162763 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.162738    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-tmp\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.162938 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.162914    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.162978 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.162921    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.170588 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.170546    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prph6\" (UniqueName: \"kubernetes.io/projected/75a1ecb9-85a5-401a-a97f-42952ce5a8ff-kube-api-access-prph6\") pod \"opendatahub-operator-controller-manager-c7885d785-j92mz\" (UID: \"75a1ecb9-85a5-401a-a97f-42952ce5a8ff\") " pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.268703 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.268605    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:18.418430 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.418402    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"]
Jul 19 14:56:18.421474 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:56:18.421448    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a1ecb9_85a5_401a_a97f_42952ce5a8ff.slice/crio-39088560911cfd5e6902a4fd8d49eb303abee7ba5be004c3596bd37a6cc46d48 WatchSource:0}: Error finding container 39088560911cfd5e6902a4fd8d49eb303abee7ba5be004c3596bd37a6cc46d48: Status 404 returned error can't find the container with id 39088560911cfd5e6902a4fd8d49eb303abee7ba5be004c3596bd37a6cc46d48
Jul 19 14:56:18.878908 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:18.878875    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz" event={"ID":"75a1ecb9-85a5-401a-a97f-42952ce5a8ff","Type":"ContainerStarted","Data":"39088560911cfd5e6902a4fd8d49eb303abee7ba5be004c3596bd37a6cc46d48"}
Jul 19 14:56:21.895974 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:21.895928    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz" event={"ID":"75a1ecb9-85a5-401a-a97f-42952ce5a8ff","Type":"ContainerStarted","Data":"a5f1abedb31cd833eeb7fd6df9c9a47b4abdaf6d40c6381cb0010dcda099d631"}
Jul 19 14:56:22.901983 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:22.901942    2572 generic.go:356] "Generic (PLEG): container finished" podID="75a1ecb9-85a5-401a-a97f-42952ce5a8ff" containerID="a5f1abedb31cd833eeb7fd6df9c9a47b4abdaf6d40c6381cb0010dcda099d631" exitCode=0
Jul 19 14:56:22.902448 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:22.902038    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz" event={"ID":"75a1ecb9-85a5-401a-a97f-42952ce5a8ff","Type":"ContainerDied","Data":"a5f1abedb31cd833eeb7fd6df9c9a47b4abdaf6d40c6381cb0010dcda099d631"}
Jul 19 14:56:23.907943 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:23.907910    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz" event={"ID":"75a1ecb9-85a5-401a-a97f-42952ce5a8ff","Type":"ContainerStarted","Data":"0ba709633c9267157b9d3fd205a552243a247b7e20fbab32dcf258ad3f8b01a3"}
Jul 19 14:56:23.908382 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:23.908053    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:23.929878 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:23.929822    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz" podStartSLOduration=2.541827591 podStartE2EDuration="6.929806694s" totalImagesPullingTime="3.378736965s" totalInitContainerRuntime="1.009242138s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:23.927420091 +0000 UTC m=+585.628868144" watchObservedRunningTime="2026-07-19 14:56:23.929806694 +0000 UTC m=+585.631254816"
Jul 19 14:56:34.915277 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:34.915245    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c7885d785-j92mz"
Jul 19 14:56:37.299663 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.299626    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wxvrm"]
Jul 19 14:56:37.302898 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.302864    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:37.306219 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.306198    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 19 14:56:37.306373 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.306224    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-xjkl8\""
Jul 19 14:56:37.313715 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.313688    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wxvrm"]
Jul 19 14:56:37.432015 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.431976    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t4rf\" (UniqueName: \"kubernetes.io/projected/d868dfab-760d-4ac1-a740-f3169366f186-kube-api-access-2t4rf\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:37.432203 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.432035    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d868dfab-760d-4ac1-a740-f3169366f186-cert\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:37.533321 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.533288    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t4rf\" (UniqueName: \"kubernetes.io/projected/d868dfab-760d-4ac1-a740-f3169366f186-kube-api-access-2t4rf\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:37.533514 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.533435    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d868dfab-760d-4ac1-a740-f3169366f186-cert\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:37.533596 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:56:37.533576    2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 19 14:56:37.533659 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:56:37.533650    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d868dfab-760d-4ac1-a740-f3169366f186-cert podName:d868dfab-760d-4ac1-a740-f3169366f186 nodeName:}" failed. No retries permitted until 2026-07-19 14:56:38.033633699 +0000 UTC m=+599.735081730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d868dfab-760d-4ac1-a740-f3169366f186-cert") pod "odh-model-controller-7998c67dc4-wxvrm" (UID: "d868dfab-760d-4ac1-a740-f3169366f186") : secret "odh-model-controller-webhook-cert" not found
Jul 19 14:56:37.544218 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.544193    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t4rf\" (UniqueName: \"kubernetes.io/projected/d868dfab-760d-4ac1-a740-f3169366f186-kube-api-access-2t4rf\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:37.804100 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.804070    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7s6gp"]
Jul 19 14:56:37.807207 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.807188    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:37.809914 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.809894    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 19 14:56:37.815777 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.815754    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7s6gp"]
Jul 19 14:56:37.937001 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.936959    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8lrr\" (UniqueName: \"kubernetes.io/projected/54167599-0655-4ff1-90af-8b9e45cc8144-kube-api-access-l8lrr\") pod \"model-serving-api-54d89bd5fc-7s6gp\" (UID: \"54167599-0655-4ff1-90af-8b9e45cc8144\") " pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:37.937193 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:37.937013    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54167599-0655-4ff1-90af-8b9e45cc8144-tls-certs\") pod \"model-serving-api-54d89bd5fc-7s6gp\" (UID: \"54167599-0655-4ff1-90af-8b9e45cc8144\") " pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:38.038058 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.038021    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8lrr\" (UniqueName: \"kubernetes.io/projected/54167599-0655-4ff1-90af-8b9e45cc8144-kube-api-access-l8lrr\") pod \"model-serving-api-54d89bd5fc-7s6gp\" (UID: \"54167599-0655-4ff1-90af-8b9e45cc8144\") " pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:38.038223 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.038066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54167599-0655-4ff1-90af-8b9e45cc8144-tls-certs\") pod \"model-serving-api-54d89bd5fc-7s6gp\" (UID: \"54167599-0655-4ff1-90af-8b9e45cc8144\") " pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:38.038223 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.038106    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d868dfab-760d-4ac1-a740-f3169366f186-cert\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:38.040551 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.040524    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/54167599-0655-4ff1-90af-8b9e45cc8144-tls-certs\") pod \"model-serving-api-54d89bd5fc-7s6gp\" (UID: \"54167599-0655-4ff1-90af-8b9e45cc8144\") " pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:38.040654 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.040600    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d868dfab-760d-4ac1-a740-f3169366f186-cert\") pod \"odh-model-controller-7998c67dc4-wxvrm\" (UID: \"d868dfab-760d-4ac1-a740-f3169366f186\") " pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:38.046684 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.046660    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8lrr\" (UniqueName: \"kubernetes.io/projected/54167599-0655-4ff1-90af-8b9e45cc8144-kube-api-access-l8lrr\") pod \"model-serving-api-54d89bd5fc-7s6gp\" (UID: \"54167599-0655-4ff1-90af-8b9e45cc8144\") " pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:38.118391 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.118356    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:38.214728 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.214696    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:38.256548 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.256528    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-7s6gp"]
Jul 19 14:56:38.258694 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:56:38.258666    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54167599_0655_4ff1_90af_8b9e45cc8144.slice/crio-83e35c93a38f3f7082ecee6e9381ff6b4a872cf1a2e6001a5c4f4d75aa5d2cb6 WatchSource:0}: Error finding container 83e35c93a38f3f7082ecee6e9381ff6b4a872cf1a2e6001a5c4f4d75aa5d2cb6: Status 404 returned error can't find the container with id 83e35c93a38f3f7082ecee6e9381ff6b4a872cf1a2e6001a5c4f4d75aa5d2cb6
Jul 19 14:56:38.347988 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.347969    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-wxvrm"]
Jul 19 14:56:38.350051 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:56:38.350026    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd868dfab_760d_4ac1_a740_f3169366f186.slice/crio-da3f9466c069630a69732cc786be0756b92acb48034e0599fe31bf1542fa8f91 WatchSource:0}: Error finding container da3f9466c069630a69732cc786be0756b92acb48034e0599fe31bf1542fa8f91: Status 404 returned error can't find the container with id da3f9466c069630a69732cc786be0756b92acb48034e0599fe31bf1542fa8f91
Jul 19 14:56:38.749020 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.748994    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:56:38.749210 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.749197    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 14:56:38.978208 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.978172    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm" event={"ID":"d868dfab-760d-4ac1-a740-f3169366f186","Type":"ContainerStarted","Data":"da3f9466c069630a69732cc786be0756b92acb48034e0599fe31bf1542fa8f91"}
Jul 19 14:56:38.979760 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:38.979703    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp" event={"ID":"54167599-0655-4ff1-90af-8b9e45cc8144","Type":"ContainerStarted","Data":"83e35c93a38f3f7082ecee6e9381ff6b4a872cf1a2e6001a5c4f4d75aa5d2cb6"}
Jul 19 14:56:39.916379 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:39.916330    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"]
Jul 19 14:56:39.922847 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:39.922824    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:39.925400 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:39.925369    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 19 14:56:39.925621 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:39.925470    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 19 14:56:39.926645 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:39.926625    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-w47qz\""
Jul 19 14:56:39.929881 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:39.929397    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"]
Jul 19 14:56:40.054537 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.054504    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212b794d-9204-48b0-a4f3-d2521b7b427d-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.054713 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.054545    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkg5c\" (UniqueName: \"kubernetes.io/projected/212b794d-9204-48b0-a4f3-d2521b7b427d-kube-api-access-jkg5c\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.054713 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.054619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/212b794d-9204-48b0-a4f3-d2521b7b427d-tmp\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.156786 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.155129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212b794d-9204-48b0-a4f3-d2521b7b427d-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.156786 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.155185    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkg5c\" (UniqueName: \"kubernetes.io/projected/212b794d-9204-48b0-a4f3-d2521b7b427d-kube-api-access-jkg5c\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.156786 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:56:40.155240    2572 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 19 14:56:40.156786 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.155243    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/212b794d-9204-48b0-a4f3-d2521b7b427d-tmp\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.156786 ip-10-0-135-98 kubenswrapper[2572]: E0719 14:56:40.155311    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/212b794d-9204-48b0-a4f3-d2521b7b427d-tls-certs podName:212b794d-9204-48b0-a4f3-d2521b7b427d nodeName:}" failed. No retries permitted until 2026-07-19 14:56:40.655290055 +0000 UTC m=+602.356738087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/212b794d-9204-48b0-a4f3-d2521b7b427d-tls-certs") pod "kube-auth-proxy-5ccf86bb4b-6csfm" (UID: "212b794d-9204-48b0-a4f3-d2521b7b427d") : secret "kube-auth-proxy-tls" not found
Jul 19 14:56:40.159272 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.159237    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/212b794d-9204-48b0-a4f3-d2521b7b427d-tmp\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.165664 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.165634    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkg5c\" (UniqueName: \"kubernetes.io/projected/212b794d-9204-48b0-a4f3-d2521b7b427d-kube-api-access-jkg5c\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.660437 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.660393    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212b794d-9204-48b0-a4f3-d2521b7b427d-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.663999 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.663974    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/212b794d-9204-48b0-a4f3-d2521b7b427d-tls-certs\") pod \"kube-auth-proxy-5ccf86bb4b-6csfm\" (UID: \"212b794d-9204-48b0-a4f3-d2521b7b427d\") " pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:40.835511 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:40.835469    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"
Jul 19 14:56:41.244301 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:41.244272    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm"]
Jul 19 14:56:41.246309 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:56:41.246283    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212b794d_9204_48b0_a4f3_d2521b7b427d.slice/crio-7ad78df541840587407bc92ce3c8ca34b4ef03f68a2ed8f513381b3c2b173a7e WatchSource:0}: Error finding container 7ad78df541840587407bc92ce3c8ca34b4ef03f68a2ed8f513381b3c2b173a7e: Status 404 returned error can't find the container with id 7ad78df541840587407bc92ce3c8ca34b4ef03f68a2ed8f513381b3c2b173a7e
Jul 19 14:56:42.003004 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.002969    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm" event={"ID":"d868dfab-760d-4ac1-a740-f3169366f186","Type":"ContainerStarted","Data":"f126c04dd792ff70fc9d749be3f6339e0c9669abc75c266d8b8338fe838f586f"}
Jul 19 14:56:42.003203 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.003153    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:42.004918 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.004861    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp" event={"ID":"54167599-0655-4ff1-90af-8b9e45cc8144","Type":"ContainerStarted","Data":"65232f0c00d238df0de1a64114de63ea25f0d7bd4d56736f68918aa9b34dac77"}
Jul 19 14:56:42.005039 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.004931    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:42.006100 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.006074    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm" event={"ID":"212b794d-9204-48b0-a4f3-d2521b7b427d","Type":"ContainerStarted","Data":"7ad78df541840587407bc92ce3c8ca34b4ef03f68a2ed8f513381b3c2b173a7e"}
Jul 19 14:56:42.019301 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.019241    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm" podStartSLOduration=2.203074958 podStartE2EDuration="5.019218919s" totalImagesPullingTime="2.816143961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:42.018086568 +0000 UTC m=+603.719534638" watchObservedRunningTime="2026-07-19 14:56:42.019218919 +0000 UTC m=+603.720666976"
Jul 19 14:56:42.037361 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:42.037285    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp" podStartSLOduration=2.183982681 podStartE2EDuration="5.037268389s" totalImagesPullingTime="2.853285708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:42.035750764 +0000 UTC m=+603.737198818" watchObservedRunningTime="2026-07-19 14:56:42.037268389 +0000 UTC m=+603.738716441"
Jul 19 14:56:44.018005 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.017970    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm" event={"ID":"212b794d-9204-48b0-a4f3-d2521b7b427d","Type":"ContainerStarted","Data":"4c7667836d84c867ddcd36cda1218ac4c84f71514db397d68fa2114f8c5e62d8"}
Jul 19 14:56:44.037866 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.037806    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5ccf86bb4b-6csfm" podStartSLOduration=2.866593543 podStartE2EDuration="5.037790414s" totalImagesPullingTime="2.171196871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:44.036188337 +0000 UTC m=+605.737636391" watchObservedRunningTime="2026-07-19 14:56:44.037790414 +0000 UTC m=+605.739238466"
Jul 19 14:56:44.434250 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.434213    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"]
Jul 19 14:56:44.437692 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.437669    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.440360 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.440327    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 19 14:56:44.440622 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.440605    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 19 14:56:44.440691 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.440673    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 19 14:56:44.440759 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.440743    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 19 14:56:44.441548 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.441532    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 19 14:56:44.441609 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.441587    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-74w5g\""
Jul 19 14:56:44.448443 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.448419    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"]
Jul 19 14:56:44.596827 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.596793    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-cert\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.597003 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.596847    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b7nm\" (UniqueName: \"kubernetes.io/projected/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-kube-api-access-7b7nm\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.597003 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.596919    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-manager-config\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.597003 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.596956    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-metrics-cert\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.698262 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.698166    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-cert\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.698262 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.698220    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b7nm\" (UniqueName: \"kubernetes.io/projected/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-kube-api-access-7b7nm\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.698262 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.698243    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-manager-config\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.698583 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.698266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-metrics-cert\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.698955 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.698933    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-manager-config\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.700831 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.700812    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-metrics-cert\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.700921 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.700853    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-cert\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.709960 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.709935    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b7nm\" (UniqueName: \"kubernetes.io/projected/30a0cbe3-dbee-4f57-bb83-7d2d704aead2-kube-api-access-7b7nm\") pod \"lws-controller-manager-5cc4ccf5bd-jdqc8\" (UID: \"30a0cbe3-dbee-4f57-bb83-7d2d704aead2\") " pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.747569 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.747536    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:44.886822 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:44.886787    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"]
Jul 19 14:56:44.890233 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:56:44.890201    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a0cbe3_dbee_4f57_bb83_7d2d704aead2.slice/crio-f7c773e383701c41d3df6c8ced8de6416dfdaa0937c284cf21b2614d64599575 WatchSource:0}: Error finding container f7c773e383701c41d3df6c8ced8de6416dfdaa0937c284cf21b2614d64599575: Status 404 returned error can't find the container with id f7c773e383701c41d3df6c8ced8de6416dfdaa0937c284cf21b2614d64599575
Jul 19 14:56:45.024920 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:45.024827    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8" event={"ID":"30a0cbe3-dbee-4f57-bb83-7d2d704aead2","Type":"ContainerStarted","Data":"f7c773e383701c41d3df6c8ced8de6416dfdaa0937c284cf21b2614d64599575"}
Jul 19 14:56:47.036908 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:47.036866    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8" event={"ID":"30a0cbe3-dbee-4f57-bb83-7d2d704aead2","Type":"ContainerStarted","Data":"a421eb4763dfd5405fb698339d03a2fab0bf96e0aed962a47a8a35de47ea0df6"}
Jul 19 14:56:47.037444 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:47.037006    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:56:47.058114 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:47.058062    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8" podStartSLOduration=1.53325723 podStartE2EDuration="3.058044384s" totalImagesPullingTime="1.524787154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:56:47.056641366 +0000 UTC m=+608.758089419" watchObservedRunningTime="2026-07-19 14:56:47.058044384 +0000 UTC m=+608.759492436"
Jul 19 14:56:53.014714 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:53.014685    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-wxvrm"
Jul 19 14:56:53.017042 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:53.017021    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-7s6gp"
Jul 19 14:56:58.043943 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:56:58.043907    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5cc4ccf5bd-jdqc8"
Jul 19 14:57:53.589809 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.589774    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"]
Jul 19 14:57:53.593171 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.593156    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.595734 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.595714    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 19 14:57:53.596797 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.596781    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wm2fr\""
Jul 19 14:57:53.596865 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.596799    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 19 14:57:53.601531 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.601494    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"]
Jul 19 14:57:53.684913 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.684879    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.685109 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.684943    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.685109 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.685029    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x487q\" (UniqueName: \"kubernetes.io/projected/a40290bb-3cf2-427c-a534-7e2c73d58c5d-kube-api-access-x487q\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.785672 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.785637    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.785866 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.785683    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x487q\" (UniqueName: \"kubernetes.io/projected/a40290bb-3cf2-427c-a534-7e2c73d58c5d-kube-api-access-x487q\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.785866 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.785722    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.786064 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.786044    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.786130 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.786076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.794466 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.794446    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x487q\" (UniqueName: \"kubernetes.io/projected/a40290bb-3cf2-427c-a534-7e2c73d58c5d-kube-api-access-x487q\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:53.904161 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:53.904081    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:54.053665 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.053638    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"]
Jul 19 14:57:54.055444 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:57:54.055409    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40290bb_3cf2_427c_a534_7e2c73d58c5d.slice/crio-28e513e86471fb2f5ec92888c818626fb451f3be6bff53a117d9e75e848e8e43 WatchSource:0}: Error finding container 28e513e86471fb2f5ec92888c818626fb451f3be6bff53a117d9e75e848e8e43: Status 404 returned error can't find the container with id 28e513e86471fb2f5ec92888c818626fb451f3be6bff53a117d9e75e848e8e43
Jul 19 14:57:54.342603 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.342564    2572 generic.go:356] "Generic (PLEG): container finished" podID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerID="9eac3f54fbfb790c3421c0b46cf610442f41c587447a8b40447350a7ede86e84" exitCode=0
Jul 19 14:57:54.342800 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.342648    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr" event={"ID":"a40290bb-3cf2-427c-a534-7e2c73d58c5d","Type":"ContainerDied","Data":"9eac3f54fbfb790c3421c0b46cf610442f41c587447a8b40447350a7ede86e84"}
Jul 19 14:57:54.342800 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.342703    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr" event={"ID":"a40290bb-3cf2-427c-a534-7e2c73d58c5d","Type":"ContainerStarted","Data":"28e513e86471fb2f5ec92888c818626fb451f3be6bff53a117d9e75e848e8e43"}
Jul 19 14:57:54.343133 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.343116    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 14:57:54.379809 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.379770    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"]
Jul 19 14:57:54.383230 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.383212    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.391218 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.391191    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"]
Jul 19 14:57:54.501742 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.501663    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.501742 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.501734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.501914 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.501757    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26b8z\" (UniqueName: \"kubernetes.io/projected/7cce55a3-59a3-441f-a612-695a2f21b3dc-kube-api-access-26b8z\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.602508 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.602475    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.602508 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.602508    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26b8z\" (UniqueName: \"kubernetes.io/projected/7cce55a3-59a3-441f-a612-695a2f21b3dc-kube-api-access-26b8z\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.602909 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.602563    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.602942 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.602926    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.602975 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.602946    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.611199 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.611177    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26b8z\" (UniqueName: \"kubernetes.io/projected/7cce55a3-59a3-441f-a612-695a2f21b3dc-kube-api-access-26b8z\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.694158 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.694121    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:54.824090 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.824066    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"]
Jul 19 14:57:54.825614 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:57:54.825588    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cce55a3_59a3_441f_a612_695a2f21b3dc.slice/crio-8ecafae9c03749f5e24a3ffddfbf4e1ab2806dfafd5986383d536d1950db31f0 WatchSource:0}: Error finding container 8ecafae9c03749f5e24a3ffddfbf4e1ab2806dfafd5986383d536d1950db31f0: Status 404 returned error can't find the container with id 8ecafae9c03749f5e24a3ffddfbf4e1ab2806dfafd5986383d536d1950db31f0
Jul 19 14:57:54.975987 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.975952    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"]
Jul 19 14:57:54.979743 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.979723    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:54.988193 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:54.988170    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"]
Jul 19 14:57:55.006351 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.006316    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.006477 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.006373    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rglsb\" (UniqueName: \"kubernetes.io/projected/2f014e40-21c6-4f61-88c9-4c3434236f75-kube-api-access-rglsb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.006477 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.006425    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.107478 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.107453    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.107611 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.107556    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.107611 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.107574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rglsb\" (UniqueName: \"kubernetes.io/projected/2f014e40-21c6-4f61-88c9-4c3434236f75-kube-api-access-rglsb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.107855 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.107834    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.107918 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.107880    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.116328 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.116303    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rglsb\" (UniqueName: \"kubernetes.io/projected/2f014e40-21c6-4f61-88c9-4c3434236f75-kube-api-access-rglsb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.289540 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.289433    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:55.351028 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.350995    2572 generic.go:356] "Generic (PLEG): container finished" podID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerID="78357353c7392fd48fb0c9af389473e94e0006ddd9779664d400a60b2f7e7c35" exitCode=0
Jul 19 14:57:55.351159 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.351130    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl" event={"ID":"7cce55a3-59a3-441f-a612-695a2f21b3dc","Type":"ContainerDied","Data":"78357353c7392fd48fb0c9af389473e94e0006ddd9779664d400a60b2f7e7c35"}
Jul 19 14:57:55.351210 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.351181    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl" event={"ID":"7cce55a3-59a3-441f-a612-695a2f21b3dc","Type":"ContainerStarted","Data":"8ecafae9c03749f5e24a3ffddfbf4e1ab2806dfafd5986383d536d1950db31f0"}
Jul 19 14:57:55.353105 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.353085    2572 generic.go:356] "Generic (PLEG): container finished" podID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerID="821b169fbae4178478f2e59fc157ace32040328791f0e93ef958cea592ed842c" exitCode=0
Jul 19 14:57:55.353266 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.353137    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr" event={"ID":"a40290bb-3cf2-427c-a534-7e2c73d58c5d","Type":"ContainerDied","Data":"821b169fbae4178478f2e59fc157ace32040328791f0e93ef958cea592ed842c"}
Jul 19 14:57:55.378028 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.377963    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"]
Jul 19 14:57:55.383384 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.383366    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.388909 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.388884    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"]
Jul 19 14:57:55.410786 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.410756    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.410908 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.410848    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.410948 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.410906    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsr55\" (UniqueName: \"kubernetes.io/projected/6cba9242-4608-4b81-bdc2-c097c9a6a27e-kube-api-access-gsr55\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.429920 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.429901    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"]
Jul 19 14:57:55.431179 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:57:55.431157    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f014e40_21c6_4f61_88c9_4c3434236f75.slice/crio-b36f360e66d680b5a8c6046cd91b35374873c3ca7e7f5d8d318e9e1087d096e2 WatchSource:0}: Error finding container b36f360e66d680b5a8c6046cd91b35374873c3ca7e7f5d8d318e9e1087d096e2: Status 404 returned error can't find the container with id b36f360e66d680b5a8c6046cd91b35374873c3ca7e7f5d8d318e9e1087d096e2
Jul 19 14:57:55.513240 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.513196    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.513465 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.513281    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.513465 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.513317    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsr55\" (UniqueName: \"kubernetes.io/projected/6cba9242-4608-4b81-bdc2-c097c9a6a27e-kube-api-access-gsr55\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.514135 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.514107    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.514260 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.514163    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.524013 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.523985    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsr55\" (UniqueName: \"kubernetes.io/projected/6cba9242-4608-4b81-bdc2-c097c9a6a27e-kube-api-access-gsr55\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.715590 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.715560    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:55.850958 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:55.850933    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"]
Jul 19 14:57:55.865876 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:57:55.865845    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cba9242_4608_4b81_bdc2_c097c9a6a27e.slice/crio-be95044e4ae7e34a95d5ab8c7e8d1b89df56055c3f11221ff2ded7977120a071 WatchSource:0}: Error finding container be95044e4ae7e34a95d5ab8c7e8d1b89df56055c3f11221ff2ded7977120a071: Status 404 returned error can't find the container with id be95044e4ae7e34a95d5ab8c7e8d1b89df56055c3f11221ff2ded7977120a071
Jul 19 14:57:56.359687 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.359651    2572 generic.go:356] "Generic (PLEG): container finished" podID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerID="92e5c1e3b4b5c952f27dcef26fadcecbd2005fb7e8f5592eddaa1c351a07f01b" exitCode=0
Jul 19 14:57:56.359838 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.359717    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr" event={"ID":"a40290bb-3cf2-427c-a534-7e2c73d58c5d","Type":"ContainerDied","Data":"92e5c1e3b4b5c952f27dcef26fadcecbd2005fb7e8f5592eddaa1c351a07f01b"}
Jul 19 14:57:56.361026 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.361006    2572 generic.go:356] "Generic (PLEG): container finished" podID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerID="f9fca7e32616f691a6a8e2a7695a7eb8850b8b37122b52ebe7bfe54337c3dafb" exitCode=0
Jul 19 14:57:56.361137 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.361075    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t" event={"ID":"6cba9242-4608-4b81-bdc2-c097c9a6a27e","Type":"ContainerDied","Data":"f9fca7e32616f691a6a8e2a7695a7eb8850b8b37122b52ebe7bfe54337c3dafb"}
Jul 19 14:57:56.361137 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.361118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t" event={"ID":"6cba9242-4608-4b81-bdc2-c097c9a6a27e","Type":"ContainerStarted","Data":"be95044e4ae7e34a95d5ab8c7e8d1b89df56055c3f11221ff2ded7977120a071"}
Jul 19 14:57:56.362560 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.362543    2572 generic.go:356] "Generic (PLEG): container finished" podID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerID="6e5a4f1d63b019fb30b674777fbfa1fefe3fa04f8effd2e9cfde94d722b11b19" exitCode=0
Jul 19 14:57:56.362664 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.362635    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l" event={"ID":"2f014e40-21c6-4f61-88c9-4c3434236f75","Type":"ContainerDied","Data":"6e5a4f1d63b019fb30b674777fbfa1fefe3fa04f8effd2e9cfde94d722b11b19"}
Jul 19 14:57:56.362712 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.362700    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l" event={"ID":"2f014e40-21c6-4f61-88c9-4c3434236f75","Type":"ContainerStarted","Data":"b36f360e66d680b5a8c6046cd91b35374873c3ca7e7f5d8d318e9e1087d096e2"}
Jul 19 14:57:56.364493 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.364477    2572 generic.go:356] "Generic (PLEG): container finished" podID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerID="d54d37cfb87e05f0500d487e7f462c0416a7f03632fc6c9954d29262f991d99c" exitCode=0
Jul 19 14:57:56.364569 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:56.364535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl" event={"ID":"7cce55a3-59a3-441f-a612-695a2f21b3dc","Type":"ContainerDied","Data":"d54d37cfb87e05f0500d487e7f462c0416a7f03632fc6c9954d29262f991d99c"}
Jul 19 14:57:57.371136 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.371107    2572 generic.go:356] "Generic (PLEG): container finished" podID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerID="8929cea76c54640defc18d29fa958d561aa4bda92ce4f70c62526deab9e967c7" exitCode=0
Jul 19 14:57:57.371550 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.371176    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t" event={"ID":"6cba9242-4608-4b81-bdc2-c097c9a6a27e","Type":"ContainerDied","Data":"8929cea76c54640defc18d29fa958d561aa4bda92ce4f70c62526deab9e967c7"}
Jul 19 14:57:57.373027 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.373004    2572 generic.go:356] "Generic (PLEG): container finished" podID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerID="465a0c0c32c12583d5ed58ed8b36b34456b518d0c986d8f2ada152b5dc3c84f9" exitCode=0
Jul 19 14:57:57.373114 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.373091    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l" event={"ID":"2f014e40-21c6-4f61-88c9-4c3434236f75","Type":"ContainerDied","Data":"465a0c0c32c12583d5ed58ed8b36b34456b518d0c986d8f2ada152b5dc3c84f9"}
Jul 19 14:57:57.375157 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.375137    2572 generic.go:356] "Generic (PLEG): container finished" podID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerID="b5492c56bae3ea0deefb99018ded865d3c823e6262e54207d7aecf9fc9fb39a4" exitCode=0
Jul 19 14:57:57.375247 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.375227    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl" event={"ID":"7cce55a3-59a3-441f-a612-695a2f21b3dc","Type":"ContainerDied","Data":"b5492c56bae3ea0deefb99018ded865d3c823e6262e54207d7aecf9fc9fb39a4"}
Jul 19 14:57:57.509042 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.509018    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:57.633201 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.633126    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-util\") pod \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") "
Jul 19 14:57:57.633201 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.633191    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x487q\" (UniqueName: \"kubernetes.io/projected/a40290bb-3cf2-427c-a534-7e2c73d58c5d-kube-api-access-x487q\") pod \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") "
Jul 19 14:57:57.633402 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.633224    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-bundle\") pod \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\" (UID: \"a40290bb-3cf2-427c-a534-7e2c73d58c5d\") "
Jul 19 14:57:57.633848 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.633820    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-bundle" (OuterVolumeSpecName: "bundle") pod "a40290bb-3cf2-427c-a534-7e2c73d58c5d" (UID: "a40290bb-3cf2-427c-a534-7e2c73d58c5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:57.635400 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.635378    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a40290bb-3cf2-427c-a534-7e2c73d58c5d-kube-api-access-x487q" (OuterVolumeSpecName: "kube-api-access-x487q") pod "a40290bb-3cf2-427c-a534-7e2c73d58c5d" (UID: "a40290bb-3cf2-427c-a534-7e2c73d58c5d"). InnerVolumeSpecName "kube-api-access-x487q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:57:57.638140 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.638121    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-util" (OuterVolumeSpecName: "util") pod "a40290bb-3cf2-427c-a534-7e2c73d58c5d" (UID: "a40290bb-3cf2-427c-a534-7e2c73d58c5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:57.734269 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.734234    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x487q\" (UniqueName: \"kubernetes.io/projected/a40290bb-3cf2-427c-a534-7e2c73d58c5d-kube-api-access-x487q\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:57.734269 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.734264    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:57.734269 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:57.734278    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a40290bb-3cf2-427c-a534-7e2c73d58c5d-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:58.382117 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.382084    2572 generic.go:356] "Generic (PLEG): container finished" podID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerID="ac7a824d52aa0181f28f163419d1a29a0b193cdf5e9b779a2c69c57096fc7117" exitCode=0
Jul 19 14:57:58.382597 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.382155    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t" event={"ID":"6cba9242-4608-4b81-bdc2-c097c9a6a27e","Type":"ContainerDied","Data":"ac7a824d52aa0181f28f163419d1a29a0b193cdf5e9b779a2c69c57096fc7117"}
Jul 19 14:57:58.384149 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.384129    2572 generic.go:356] "Generic (PLEG): container finished" podID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerID="441554b1a7b9f6ff8f88d882076f9baa77cc107e471f7458e07481ae530c618b" exitCode=0
Jul 19 14:57:58.384237 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.384213    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l" event={"ID":"2f014e40-21c6-4f61-88c9-4c3434236f75","Type":"ContainerDied","Data":"441554b1a7b9f6ff8f88d882076f9baa77cc107e471f7458e07481ae530c618b"}
Jul 19 14:57:58.385909 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.385893    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr"
Jul 19 14:57:58.385993 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.385924    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr" event={"ID":"a40290bb-3cf2-427c-a534-7e2c73d58c5d","Type":"ContainerDied","Data":"28e513e86471fb2f5ec92888c818626fb451f3be6bff53a117d9e75e848e8e43"}
Jul 19 14:57:58.385993 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.385953    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28e513e86471fb2f5ec92888c818626fb451f3be6bff53a117d9e75e848e8e43"
Jul 19 14:57:58.513859 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.513825    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:58.641091 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.641003    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-bundle\") pod \"7cce55a3-59a3-441f-a612-695a2f21b3dc\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") "
Jul 19 14:57:58.641091 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.641077    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26b8z\" (UniqueName: \"kubernetes.io/projected/7cce55a3-59a3-441f-a612-695a2f21b3dc-kube-api-access-26b8z\") pod \"7cce55a3-59a3-441f-a612-695a2f21b3dc\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") "
Jul 19 14:57:58.641306 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.641148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-util\") pod \"7cce55a3-59a3-441f-a612-695a2f21b3dc\" (UID: \"7cce55a3-59a3-441f-a612-695a2f21b3dc\") "
Jul 19 14:57:58.641587 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.641561    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-bundle" (OuterVolumeSpecName: "bundle") pod "7cce55a3-59a3-441f-a612-695a2f21b3dc" (UID: "7cce55a3-59a3-441f-a612-695a2f21b3dc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:58.643250 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.643229    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cce55a3-59a3-441f-a612-695a2f21b3dc-kube-api-access-26b8z" (OuterVolumeSpecName: "kube-api-access-26b8z") pod "7cce55a3-59a3-441f-a612-695a2f21b3dc" (UID: "7cce55a3-59a3-441f-a612-695a2f21b3dc"). InnerVolumeSpecName "kube-api-access-26b8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:57:58.646187 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.646169    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-util" (OuterVolumeSpecName: "util") pod "7cce55a3-59a3-441f-a612-695a2f21b3dc" (UID: "7cce55a3-59a3-441f-a612-695a2f21b3dc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:58.742039 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.742011    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:58.742039 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.742035    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7cce55a3-59a3-441f-a612-695a2f21b3dc-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:58.742039 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:58.742045    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26b8z\" (UniqueName: \"kubernetes.io/projected/7cce55a3-59a3-441f-a612-695a2f21b3dc-kube-api-access-26b8z\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:59.397936 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.397902    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl"
Jul 19 14:57:59.397936 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.397923    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl" event={"ID":"7cce55a3-59a3-441f-a612-695a2f21b3dc","Type":"ContainerDied","Data":"8ecafae9c03749f5e24a3ffddfbf4e1ab2806dfafd5986383d536d1950db31f0"}
Jul 19 14:57:59.398461 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.397987    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ecafae9c03749f5e24a3ffddfbf4e1ab2806dfafd5986383d536d1950db31f0"
Jul 19 14:57:59.549406 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.549386    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:57:59.551499 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.551481    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:57:59.649464 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.649381    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rglsb\" (UniqueName: \"kubernetes.io/projected/2f014e40-21c6-4f61-88c9-4c3434236f75-kube-api-access-rglsb\") pod \"2f014e40-21c6-4f61-88c9-4c3434236f75\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") "
Jul 19 14:57:59.649464 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.649424    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-util\") pod \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") "
Jul 19 14:57:59.649464 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.649446    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gsr55\" (UniqueName: \"kubernetes.io/projected/6cba9242-4608-4b81-bdc2-c097c9a6a27e-kube-api-access-gsr55\") pod \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") "
Jul 19 14:57:59.649736 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.649568    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-bundle\") pod \"2f014e40-21c6-4f61-88c9-4c3434236f75\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") "
Jul 19 14:57:59.649736 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.649630    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-bundle\") pod \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\" (UID: \"6cba9242-4608-4b81-bdc2-c097c9a6a27e\") "
Jul 19 14:57:59.649736 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.649656    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-util\") pod \"2f014e40-21c6-4f61-88c9-4c3434236f75\" (UID: \"2f014e40-21c6-4f61-88c9-4c3434236f75\") "
Jul 19 14:57:59.650203 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.650146    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-bundle" (OuterVolumeSpecName: "bundle") pod "6cba9242-4608-4b81-bdc2-c097c9a6a27e" (UID: "6cba9242-4608-4b81-bdc2-c097c9a6a27e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:59.650203 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.650180    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-bundle" (OuterVolumeSpecName: "bundle") pod "2f014e40-21c6-4f61-88c9-4c3434236f75" (UID: "2f014e40-21c6-4f61-88c9-4c3434236f75"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:59.651639 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.651611    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f014e40-21c6-4f61-88c9-4c3434236f75-kube-api-access-rglsb" (OuterVolumeSpecName: "kube-api-access-rglsb") pod "2f014e40-21c6-4f61-88c9-4c3434236f75" (UID: "2f014e40-21c6-4f61-88c9-4c3434236f75"). InnerVolumeSpecName "kube-api-access-rglsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:57:59.651725 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.651706    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cba9242-4608-4b81-bdc2-c097c9a6a27e-kube-api-access-gsr55" (OuterVolumeSpecName: "kube-api-access-gsr55") pod "6cba9242-4608-4b81-bdc2-c097c9a6a27e" (UID: "6cba9242-4608-4b81-bdc2-c097c9a6a27e"). InnerVolumeSpecName "kube-api-access-gsr55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 14:57:59.655501 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.655471    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-util" (OuterVolumeSpecName: "util") pod "2f014e40-21c6-4f61-88c9-4c3434236f75" (UID: "2f014e40-21c6-4f61-88c9-4c3434236f75"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:59.656267 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.656246    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-util" (OuterVolumeSpecName: "util") pod "6cba9242-4608-4b81-bdc2-c097c9a6a27e" (UID: "6cba9242-4608-4b81-bdc2-c097c9a6a27e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 14:57:59.751211 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.751179    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:59.751211 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.751204    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-bundle\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:59.751211 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.751212    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f014e40-21c6-4f61-88c9-4c3434236f75-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:59.751211 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.751220    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rglsb\" (UniqueName: \"kubernetes.io/projected/2f014e40-21c6-4f61-88c9-4c3434236f75-kube-api-access-rglsb\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:59.751461 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.751229    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6cba9242-4608-4b81-bdc2-c097c9a6a27e-util\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:57:59.751461 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:57:59.751237    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsr55\" (UniqueName: \"kubernetes.io/projected/6cba9242-4608-4b81-bdc2-c097c9a6a27e-kube-api-access-gsr55\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 14:58:00.404486 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:00.404456    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t" event={"ID":"6cba9242-4608-4b81-bdc2-c097c9a6a27e","Type":"ContainerDied","Data":"be95044e4ae7e34a95d5ab8c7e8d1b89df56055c3f11221ff2ded7977120a071"}
Jul 19 14:58:00.404902 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:00.404501    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be95044e4ae7e34a95d5ab8c7e8d1b89df56055c3f11221ff2ded7977120a071"
Jul 19 14:58:00.404902 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:00.404469    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t"
Jul 19 14:58:00.406428 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:00.406406    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l" event={"ID":"2f014e40-21c6-4f61-88c9-4c3434236f75","Type":"ContainerDied","Data":"b36f360e66d680b5a8c6046cd91b35374873c3ca7e7f5d8d318e9e1087d096e2"}
Jul 19 14:58:00.406530 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:00.406443    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b36f360e66d680b5a8c6046cd91b35374873c3ca7e7f5d8d318e9e1087d096e2"
Jul 19 14:58:00.406530 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:00.406414    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l"
Jul 19 14:58:17.415187 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415122    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"]
Jul 19 14:58:17.415659 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415640    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="pull"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415661    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="pull"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415673    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="util"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415678    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="util"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415684    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="pull"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415689    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="pull"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415696    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="pull"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415701    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="pull"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415708    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="extract"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415713    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="extract"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415721    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="util"
Jul 19 14:58:17.415720 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415726    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="util"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415731    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415737    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415743    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="util"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415748    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="util"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415755    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="pull"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415759    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="pull"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415765    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="util"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415769    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="util"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415774    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415778    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415792    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415797    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415860    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f014e40-21c6-4f61-88c9-4c3434236f75" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415867    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cce55a3-59a3-441f-a612-695a2f21b3dc" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415872    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a40290bb-3cf2-427c-a534-7e2c73d58c5d" containerName="extract"
Jul 19 14:58:17.416184 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.415879    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cba9242-4608-4b81-bdc2-c097c9a6a27e" containerName="extract"
Jul 19 14:58:17.418816 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.418800    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.421286 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.421260    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 19 14:58:17.421448 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.421394    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 19 14:58:17.421448 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.421403    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 19 14:58:17.421448 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.421410    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 19 14:58:17.422426 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.422410    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wm2fr\""
Jul 19 14:58:17.428148 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.428130    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"]
Jul 19 14:58:17.500134 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.500099    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d9d6a028-4308-450d-bc2e-2a768c7bf88a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.500288 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.500188    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln8km\" (UniqueName: \"kubernetes.io/projected/d9d6a028-4308-450d-bc2e-2a768c7bf88a-kube-api-access-ln8km\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.500288 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.500220    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d6a028-4308-450d-bc2e-2a768c7bf88a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.601221 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.601186    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln8km\" (UniqueName: \"kubernetes.io/projected/d9d6a028-4308-450d-bc2e-2a768c7bf88a-kube-api-access-ln8km\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.601393 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.601232    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d6a028-4308-450d-bc2e-2a768c7bf88a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.601393 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.601269    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d9d6a028-4308-450d-bc2e-2a768c7bf88a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.601877 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.601858    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d9d6a028-4308-450d-bc2e-2a768c7bf88a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.603868 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.603846    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d6a028-4308-450d-bc2e-2a768c7bf88a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.609295 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.609269    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln8km\" (UniqueName: \"kubernetes.io/projected/d9d6a028-4308-450d-bc2e-2a768c7bf88a-kube-api-access-ln8km\") pod \"kuadrant-console-plugin-66b5774985-554nf\" (UID: \"d9d6a028-4308-450d-bc2e-2a768c7bf88a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.728141 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.728064    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"
Jul 19 14:58:17.858782 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:17.858755    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-554nf"]
Jul 19 14:58:17.860658 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:58:17.860626    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9d6a028_4308_450d_bc2e_2a768c7bf88a.slice/crio-7af28af5369226bea3939a10d06efba954dead32fedec32540051226f922c31b WatchSource:0}: Error finding container 7af28af5369226bea3939a10d06efba954dead32fedec32540051226f922c31b: Status 404 returned error can't find the container with id 7af28af5369226bea3939a10d06efba954dead32fedec32540051226f922c31b
Jul 19 14:58:18.494099 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:18.494060    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf" event={"ID":"d9d6a028-4308-450d-bc2e-2a768c7bf88a","Type":"ContainerStarted","Data":"7af28af5369226bea3939a10d06efba954dead32fedec32540051226f922c31b"}
Jul 19 14:58:39.617095 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:39.617060    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf" event={"ID":"d9d6a028-4308-450d-bc2e-2a768c7bf88a","Type":"ContainerStarted","Data":"3c1d1500bb17b2d565938ed11daed6967e1d7e74f00fc4e09dfe8e12aaac5542"}
Jul 19 14:58:39.634498 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:58:39.634432    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-554nf" podStartSLOduration=1.8900175620000002 podStartE2EDuration="22.634411958s" totalImagesPullingTime="20.744394396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:58:39.633778193 +0000 UTC m=+721.335226246" watchObservedRunningTime="2026-07-19 14:58:39.634411958 +0000 UTC m=+721.335860011"
Jul 19 14:59:04.875251 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:04.875211    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 14:59:05.333213 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.333178    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 14:59:05.333396 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.333227    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 14:59:05.333396 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.333301    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.335949 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.335913    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 19 14:59:05.431675 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.431635    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92d8n\" (UniqueName: \"kubernetes.io/projected/191e16a3-ae71-49f7-9aab-77293ebb28c1-kube-api-access-92d8n\") pod \"limitador-limitador-54d6d988d5-4c2hn\" (UID: \"191e16a3-ae71-49f7-9aab-77293ebb28c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.431675 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.431679    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/191e16a3-ae71-49f7-9aab-77293ebb28c1-config-file\") pod \"limitador-limitador-54d6d988d5-4c2hn\" (UID: \"191e16a3-ae71-49f7-9aab-77293ebb28c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.532691 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.532657    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92d8n\" (UniqueName: \"kubernetes.io/projected/191e16a3-ae71-49f7-9aab-77293ebb28c1-kube-api-access-92d8n\") pod \"limitador-limitador-54d6d988d5-4c2hn\" (UID: \"191e16a3-ae71-49f7-9aab-77293ebb28c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.532691 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.532696    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/191e16a3-ae71-49f7-9aab-77293ebb28c1-config-file\") pod \"limitador-limitador-54d6d988d5-4c2hn\" (UID: \"191e16a3-ae71-49f7-9aab-77293ebb28c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.533370 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.533322    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/191e16a3-ae71-49f7-9aab-77293ebb28c1-config-file\") pod \"limitador-limitador-54d6d988d5-4c2hn\" (UID: \"191e16a3-ae71-49f7-9aab-77293ebb28c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.541838 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.541811    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92d8n\" (UniqueName: \"kubernetes.io/projected/191e16a3-ae71-49f7-9aab-77293ebb28c1-kube-api-access-92d8n\") pod \"limitador-limitador-54d6d988d5-4c2hn\" (UID: \"191e16a3-ae71-49f7-9aab-77293ebb28c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.643186 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.643098    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:05.855574 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:05.855543    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 14:59:05.857158 ip-10-0-135-98 kubenswrapper[2572]: W0719 14:59:05.857123    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191e16a3_ae71_49f7_9aab_77293ebb28c1.slice/crio-b83589cc63d4c7aa1284a217169101a5780e284864de9a9a7bd9bc1fd977e770 WatchSource:0}: Error finding container b83589cc63d4c7aa1284a217169101a5780e284864de9a9a7bd9bc1fd977e770: Status 404 returned error can't find the container with id b83589cc63d4c7aa1284a217169101a5780e284864de9a9a7bd9bc1fd977e770
Jul 19 14:59:06.750236 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:06.750205    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn" event={"ID":"191e16a3-ae71-49f7-9aab-77293ebb28c1","Type":"ContainerStarted","Data":"b83589cc63d4c7aa1284a217169101a5780e284864de9a9a7bd9bc1fd977e770"}
Jul 19 14:59:10.775256 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:10.775212    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn" event={"ID":"191e16a3-ae71-49f7-9aab-77293ebb28c1","Type":"ContainerStarted","Data":"2f5e1b3f56b654df4f35f5d9551a777b745986cb9e3d537215bc32be33249766"}
Jul 19 14:59:10.775806 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:10.775275    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 14:59:10.791196 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:10.791146    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn" podStartSLOduration=2.873475513 podStartE2EDuration="6.791132432s" totalImagesPullingTime="3.917656919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 14:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 14:59:10.790769922 +0000 UTC m=+752.492217975" watchObservedRunningTime="2026-07-19 14:59:10.791132432 +0000 UTC m=+752.492580485"
Jul 19 14:59:21.782204 ip-10-0-135-98 kubenswrapper[2572]: I0719 14:59:21.782173    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"
Jul 19 15:00:21.304601 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:00:21.304560    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:01:02.265572 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:01:02.265536    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:01:08.051516 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:01:08.051488    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:01:20.165928 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:01:20.165885    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:01:38.785174 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:01:38.785138    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:01:38.786491 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:01:38.786469    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:02:03.651221 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:03.651184    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:02:28.562051 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:28.562013    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:02:34.477980 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.477945    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"]
Jul 19 15:02:34.481135 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.481115    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.483860 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.483835    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 19 15:02:34.483981 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.483887    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 19 15:02:34.484868 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.484842    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bfqdt\""
Jul 19 15:02:34.484979 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.484868    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 19 15:02:34.490062 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.490041    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"]
Jul 19 15:02:34.606602 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.606574    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.606742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.606607    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.606742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.606639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.606742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.606681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm5pw\" (UniqueName: \"kubernetes.io/projected/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-kube-api-access-dm5pw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.606742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.606714    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.606871 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.606750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707434 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707405    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707561 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707439    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707561 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707468    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707640 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707584    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm5pw\" (UniqueName: \"kubernetes.io/projected/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-kube-api-access-dm5pw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707640 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707626    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707731 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707798 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707780    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707849 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707816    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.707951 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.707934    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.709840 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.709820    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.709917 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.709885    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.715965 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.715946    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm5pw\" (UniqueName: \"kubernetes.io/projected/a3aefd58-d7e5-49f6-94a6-4d199e88d7af-kube-api-access-dm5pw\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn\" (UID: \"a3aefd58-d7e5-49f6-94a6-4d199e88d7af\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.791700 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.791625    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:34.919641 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:34.919612    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"]
Jul 19 15:02:34.921078 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:02:34.921052    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3aefd58_d7e5_49f6_94a6_4d199e88d7af.slice/crio-3fd3f739d06d3de7c73eeb667baef0a7aa4cc7791c66f1be9cab58b14bfdb437 WatchSource:0}: Error finding container 3fd3f739d06d3de7c73eeb667baef0a7aa4cc7791c66f1be9cab58b14bfdb437: Status 404 returned error can't find the container with id 3fd3f739d06d3de7c73eeb667baef0a7aa4cc7791c66f1be9cab58b14bfdb437
Jul 19 15:02:35.791044 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:35.791008    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn" event={"ID":"a3aefd58-d7e5-49f6-94a6-4d199e88d7af","Type":"ContainerStarted","Data":"3fd3f739d06d3de7c73eeb667baef0a7aa4cc7791c66f1be9cab58b14bfdb437"}
Jul 19 15:02:36.796653 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:36.796621    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn" event={"ID":"a3aefd58-d7e5-49f6-94a6-4d199e88d7af","Type":"ContainerStarted","Data":"acb2240a00c16a2a7fb981ecd26e7c88ce130d286b562901e16e85bfd7b5d9c5"}
Jul 19 15:02:36.813995 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:36.813813    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn" podStartSLOduration=1.814772534 podStartE2EDuration="2.813791128s" totalImagesPullingTime="999.018594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:02:36.813424037 +0000 UTC m=+958.514872090" watchObservedRunningTime="2026-07-19 15:02:36.813791128 +0000 UTC m=+958.515239181"
Jul 19 15:02:37.855969 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:37.855935    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:02:44.791889 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:44.791857    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:44.792498 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:44.791903    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:44.803556 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:44.803536    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:02:44.843944 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:02:44.843919    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn"
Jul 19 15:04:04.353969 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:04:04.353936    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:04:15.466881 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:04:15.466836    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:04:26.359611 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:04:26.359571    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:06:05.460280 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:06:05.460196    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:06:16.256308 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:06:16.256269    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:06:38.819690 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:06:38.819660    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:06:38.821765 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:06:38.821743    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:06:49.997923 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:06:49.997888    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:06:57.659552 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:06:57.659518    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:07:08.851721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:07:08.851683    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:07:16.053321 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:07:16.053278    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:07:25.556663 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:07:25.556623    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:07:36.858304 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:07:36.858220    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:07:47.346916 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:07:47.346880    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:07:57.049266 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:07:57.049231    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:08:22.063925 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:08:22.063887    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:08:30.952935 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:08:30.952894    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:08:54.255186 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:08:54.255151    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:09:06.063458 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:09:06.063376    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:09:20.858877 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:09:20.858837    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:09:30.852665 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:09:30.852627    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:09:46.854169 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:09:46.854134    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:11:38.860264 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:11:38.860234    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:11:38.862621 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:11:38.862598    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:11:46.659095 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:11:46.659061    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:11:58.264096 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:11:58.264055    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:12:08.557741 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:12:08.557705    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:12:19.659735 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:12:19.659691    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:12:30.860518 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:12:30.860481    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:12:41.554853 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:12:41.554813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:12:50.354583 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:12:50.354547    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:13:01.146368 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:13:01.146322    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:13:11.364732 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:13:11.364696    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:13:21.355461 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:13:21.355424    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:13:31.655000 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:13:31.654913    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:13:56.456937 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:13:56.456900    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:14:03.762565 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:03.762527    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:14:17.591317 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.591279    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"]
Jul 19 15:14:17.594652 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.594630    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.597593 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.597572    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-79056b85-kserve-self-signed-certs\""
Jul 19 15:14:17.615511 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.615476    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"]
Jul 19 15:14:17.670556 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.670512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2438c-ffb1-4d42-acde-6944c20d6435-tls-certs\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.670742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.670566    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-dshm\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.670742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.670584    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-model-cache\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.670742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.670640    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6zt4\" (UniqueName: \"kubernetes.io/projected/3dc2438c-ffb1-4d42-acde-6944c20d6435-kube-api-access-b6zt4\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.670742 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.670734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-home\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.670876 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.670757    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-tmp-dir\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.771688 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.771655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-dshm\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.771688 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.771689    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-model-cache\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.771913 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.771713    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6zt4\" (UniqueName: \"kubernetes.io/projected/3dc2438c-ffb1-4d42-acde-6944c20d6435-kube-api-access-b6zt4\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.771913 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.771749    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-home\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.771913 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.771766    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-tmp-dir\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.771913 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.771814    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2438c-ffb1-4d42-acde-6944c20d6435-tls-certs\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.772197 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.772177    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-model-cache\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.772257 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.772231    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-home\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.772310 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.772283    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-tmp-dir\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.774204 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.774173    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-dshm\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.774441 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.774423    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2438c-ffb1-4d42-acde-6944c20d6435-tls-certs\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.783260 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.783231    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6zt4\" (UniqueName: \"kubernetes.io/projected/3dc2438c-ffb1-4d42-acde-6944c20d6435-kube-api-access-b6zt4\") pod \"e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") " pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:17.905976 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:17.905888    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:18.059503 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:18.059473    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"]
Jul 19 15:14:18.060992 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:14:18.060967    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dc2438c_ffb1_4d42_acde_6944c20d6435.slice/crio-1960e8a7d25799ae2421178088a776f843df1bb7bd59d5f8ca2bb59b3c5f87bb WatchSource:0}: Error finding container 1960e8a7d25799ae2421178088a776f843df1bb7bd59d5f8ca2bb59b3c5f87bb: Status 404 returned error can't find the container with id 1960e8a7d25799ae2421178088a776f843df1bb7bd59d5f8ca2bb59b3c5f87bb
Jul 19 15:14:18.343516 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:18.343477    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr" event={"ID":"3dc2438c-ffb1-4d42-acde-6944c20d6435","Type":"ContainerStarted","Data":"01b73c3be6f5b855fff55a259cb6001cbde76fc1c3a1d1c572758d1abb2204b2"}
Jul 19 15:14:18.343695 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:18.343526    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr" event={"ID":"3dc2438c-ffb1-4d42-acde-6944c20d6435","Type":"ContainerStarted","Data":"1960e8a7d25799ae2421178088a776f843df1bb7bd59d5f8ca2bb59b3c5f87bb"}
Jul 19 15:14:18.361744 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:18.361689    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr" podStartSLOduration=1.361673179 podStartE2EDuration="1.361673179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:14:18.360575561 +0000 UTC m=+1660.062023614" watchObservedRunningTime="2026-07-19 15:14:18.361673179 +0000 UTC m=+1660.063121232"
Jul 19 15:14:20.552639 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:20.552595    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:14:27.906892 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:27.906829    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:27.907496 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:27.906929    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:27.920075 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:27.920047    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:28.405544 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:28.405508    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:14:37.258248 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:14:37.258212    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:15:08.420966 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:08.420873    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"]
Jul 19 15:15:12.046642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:12.046601    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:15:23.442974 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.442903    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr" podUID="3dc2438c-ffb1-4d42-acde-6944c20d6435" containerName="main" containerID="cri-o://01b73c3be6f5b855fff55a259cb6001cbde76fc1c3a1d1c572758d1abb2204b2" gracePeriod=45
Jul 19 15:15:23.679611 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.679582    2572 generic.go:356] "Generic (PLEG): container finished" podID="3dc2438c-ffb1-4d42-acde-6944c20d6435" containerID="01b73c3be6f5b855fff55a259cb6001cbde76fc1c3a1d1c572758d1abb2204b2" exitCode=0
Jul 19 15:15:23.679759 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.679669    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr" event={"ID":"3dc2438c-ffb1-4d42-acde-6944c20d6435","Type":"ContainerDied","Data":"01b73c3be6f5b855fff55a259cb6001cbde76fc1c3a1d1c572758d1abb2204b2"}
Jul 19 15:15:23.679759 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.679720    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr" event={"ID":"3dc2438c-ffb1-4d42-acde-6944c20d6435","Type":"ContainerDied","Data":"1960e8a7d25799ae2421178088a776f843df1bb7bd59d5f8ca2bb59b3c5f87bb"}
Jul 19 15:15:23.679759 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.679738    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1960e8a7d25799ae2421178088a776f843df1bb7bd59d5f8ca2bb59b3c5f87bb"
Jul 19 15:15:23.693856 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.693788    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:15:23.794843 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.794805    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6zt4\" (UniqueName: \"kubernetes.io/projected/3dc2438c-ffb1-4d42-acde-6944c20d6435-kube-api-access-b6zt4\") pod \"3dc2438c-ffb1-4d42-acde-6944c20d6435\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") "
Jul 19 15:15:23.795048 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.794865    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-tmp-dir\") pod \"3dc2438c-ffb1-4d42-acde-6944c20d6435\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") "
Jul 19 15:15:23.795048 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.794914    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-dshm\") pod \"3dc2438c-ffb1-4d42-acde-6944c20d6435\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") "
Jul 19 15:15:23.795048 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.794948    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-model-cache\") pod \"3dc2438c-ffb1-4d42-acde-6944c20d6435\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") "
Jul 19 15:15:23.795202 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795047    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2438c-ffb1-4d42-acde-6944c20d6435-tls-certs\") pod \"3dc2438c-ffb1-4d42-acde-6944c20d6435\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") "
Jul 19 15:15:23.795202 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795069    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-home\") pod \"3dc2438c-ffb1-4d42-acde-6944c20d6435\" (UID: \"3dc2438c-ffb1-4d42-acde-6944c20d6435\") "
Jul 19 15:15:23.795202 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795155    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3dc2438c-ffb1-4d42-acde-6944c20d6435" (UID: "3dc2438c-ffb1-4d42-acde-6944c20d6435"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:23.795394 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795283    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-model-cache" (OuterVolumeSpecName: "model-cache") pod "3dc2438c-ffb1-4d42-acde-6944c20d6435" (UID: "3dc2438c-ffb1-4d42-acde-6944c20d6435"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:23.795394 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795327    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-home" (OuterVolumeSpecName: "home") pod "3dc2438c-ffb1-4d42-acde-6944c20d6435" (UID: "3dc2438c-ffb1-4d42-acde-6944c20d6435"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:23.795500 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795423    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:15:23.795500 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.795443    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:15:23.797437 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.797410    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-dshm" (OuterVolumeSpecName: "dshm") pod "3dc2438c-ffb1-4d42-acde-6944c20d6435" (UID: "3dc2438c-ffb1-4d42-acde-6944c20d6435"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:15:23.797560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.797525    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc2438c-ffb1-4d42-acde-6944c20d6435-kube-api-access-b6zt4" (OuterVolumeSpecName: "kube-api-access-b6zt4") pod "3dc2438c-ffb1-4d42-acde-6944c20d6435" (UID: "3dc2438c-ffb1-4d42-acde-6944c20d6435"). InnerVolumeSpecName "kube-api-access-b6zt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:15:23.797626 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.797606    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dc2438c-ffb1-4d42-acde-6944c20d6435-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3dc2438c-ffb1-4d42-acde-6944c20d6435" (UID: "3dc2438c-ffb1-4d42-acde-6944c20d6435"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:15:23.895914 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.895870    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dc2438c-ffb1-4d42-acde-6944c20d6435-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:15:23.895914 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.895906    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:15:23.895914 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.895917    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6zt4\" (UniqueName: \"kubernetes.io/projected/3dc2438c-ffb1-4d42-acde-6944c20d6435-kube-api-access-b6zt4\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:15:23.895914 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:23.895926    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dc2438c-ffb1-4d42-acde-6944c20d6435-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:15:24.358265 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:24.358229    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:15:24.685006 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:24.684917    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"
Jul 19 15:15:24.713081 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:24.713033    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"]
Jul 19 15:15:24.714103 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:24.714070    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-79056b85-kserve-678d968f95-6xtdr"]
Jul 19 15:15:24.786423 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:24.786390    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc2438c-ffb1-4d42-acde-6944c20d6435" path="/var/lib/kubelet/pods/3dc2438c-ffb1-4d42-acde-6944c20d6435/volumes"
Jul 19 15:15:30.764004 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:30.763956    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:15:47.361269 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:47.361231    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:15:58.355710 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:15:58.355663    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:16:07.255270 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:07.255227    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:16:17.175531 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:17.175490    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:16:26.160288 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:26.160249    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:16:35.450566 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:35.450458    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:16:38.898894 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:38.898863    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:16:38.902159 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:38.902132    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:16:48.261984 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:48.261945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:16:58.170665 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:16:58.170624    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:17:07.553398 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:17:07.553357    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:17:18.786106 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:17:18.786069    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:17:29.352392 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:17:29.352327    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:17:35.955250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:17:35.955205    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:17:52.859632 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:17:52.859579    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:18:04.453264 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:18:04.453163    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:18:13.550702 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:18:13.550662    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:18:26.555286 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:18:26.555248    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:18:35.550594 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:18:35.550555    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:18:46.656133 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:18:46.656090    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:18:56.456509 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:18:56.456472    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:19:28.657078 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:19:28.657038    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:19:39.550769 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:19:39.550684    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:20:12.049457 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:20:12.049420    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:20:23.154570 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:20:23.154531    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:20:38.933329 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:20:38.933299    2572 scope.go:117] "RemoveContainer" containerID="01b73c3be6f5b855fff55a259cb6001cbde76fc1c3a1d1c572758d1abb2204b2"
Jul 19 15:20:55.859759 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:20:55.859720    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:21:04.954803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:21:04.954718    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:21:38.933999 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:21:38.933968    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:21:38.938377 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:21:38.938351    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:21:40.263586 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:21:40.263553    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:23:08.058382 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:23:08.058270    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:23:17.959056 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:23:17.959015    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:23:28.262613 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:23:28.262580    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:23:47.858081 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:23:47.858042    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:24:25.054386 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:25.054347    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:24:37.691082 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.691048    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"]
Jul 19 15:24:37.691516 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.691438    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc2438c-ffb1-4d42-acde-6944c20d6435" containerName="main"
Jul 19 15:24:37.691516 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.691455    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc2438c-ffb1-4d42-acde-6944c20d6435" containerName="main"
Jul 19 15:24:37.691624 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.691603    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dc2438c-ffb1-4d42-acde-6944c20d6435" containerName="main"
Jul 19 15:24:37.694749 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.694731    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.697558 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.697521    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-28015a4b\"/\"openshift-service-ca.crt\""
Jul 19 15:24:37.697558 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.697552    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-28015a4b\"/\"kube-root-ca.crt\""
Jul 19 15:24:37.699043 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.699002    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-28015a4b\"/\"default-dockercfg-tv5cz\""
Jul 19 15:24:37.699233 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.699045    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-28015a4b\"/\"e2e-lifecycle-model-28015a4b-kserve-self-signed-certs\""
Jul 19 15:24:37.704371 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.704325    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"]
Jul 19 15:24:37.760595 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.760564    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-tmp-dir\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.760758 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.760610    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-dshm\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.760758 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.760641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-model-cache\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.760758 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.760681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-home\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.760758 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.760698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmpzn\" (UniqueName: \"kubernetes.io/projected/c31867ec-4612-4cb1-9321-10e892416814-kube-api-access-wmpzn\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.760891 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.760821    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31867ec-4612-4cb1-9321-10e892416814-tls-certs\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.861834 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.861799    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31867ec-4612-4cb1-9321-10e892416814-tls-certs\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862030 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862010    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-tmp-dir\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862082 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862061    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-dshm\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862124 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862103    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-model-cache\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862170 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862138    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-home\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862226 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862167    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmpzn\" (UniqueName: \"kubernetes.io/projected/c31867ec-4612-4cb1-9321-10e892416814-kube-api-access-wmpzn\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862486 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862462    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-tmp-dir\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862605 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862565    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-model-cache\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.862659 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.862600    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-home\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.864318 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.864301    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-dshm\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.864573 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.864558    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31867ec-4612-4cb1-9321-10e892416814-tls-certs\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:37.870063 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:37.870039    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmpzn\" (UniqueName: \"kubernetes.io/projected/c31867ec-4612-4cb1-9321-10e892416814-kube-api-access-wmpzn\") pod \"e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") " pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:38.006465 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:38.006371    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:38.141395 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:38.141371    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"]
Jul 19 15:24:38.143475 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:24:38.143449    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc31867ec_4612_4cb1_9321_10e892416814.slice/crio-560e4fd467c6fa643319f6f1c8be83f4142ee90f6a6f06a4f23c43f9919fcd23 WatchSource:0}: Error finding container 560e4fd467c6fa643319f6f1c8be83f4142ee90f6a6f06a4f23c43f9919fcd23: Status 404 returned error can't find the container with id 560e4fd467c6fa643319f6f1c8be83f4142ee90f6a6f06a4f23c43f9919fcd23
Jul 19 15:24:38.530434 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:38.530399    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb" event={"ID":"c31867ec-4612-4cb1-9321-10e892416814","Type":"ContainerStarted","Data":"c439355a28bf815e54693f362431bde7a207f038b04b1023a77ee53b736c1e2c"}
Jul 19 15:24:38.530607 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:38.530445    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb" event={"ID":"c31867ec-4612-4cb1-9321-10e892416814","Type":"ContainerStarted","Data":"560e4fd467c6fa643319f6f1c8be83f4142ee90f6a6f06a4f23c43f9919fcd23"}
Jul 19 15:24:38.547523 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:38.547475    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb" podStartSLOduration=1.547461303 podStartE2EDuration="1.547461303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:24:38.545476818 +0000 UTC m=+2280.246924875" watchObservedRunningTime="2026-07-19 15:24:38.547461303 +0000 UTC m=+2280.248909358"
Jul 19 15:24:40.609225 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:40.609188    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:24:48.007316 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:48.007284    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:48.007744 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:48.007452    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:48.020379 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:48.020330    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:24:48.598116 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:24:48.598085    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:25:10.561872 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:10.561836    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:25:31.154744 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:31.154704    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:25:39.953726 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:39.953690    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:25:44.689409 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:44.689368    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"]
Jul 19 15:25:59.709796 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.709729    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb" podUID="c31867ec-4612-4cb1-9321-10e892416814" containerName="main" containerID="cri-o://c439355a28bf815e54693f362431bde7a207f038b04b1023a77ee53b736c1e2c" gracePeriod=45
Jul 19 15:25:59.943373 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.943325    2572 generic.go:356] "Generic (PLEG): container finished" podID="c31867ec-4612-4cb1-9321-10e892416814" containerID="c439355a28bf815e54693f362431bde7a207f038b04b1023a77ee53b736c1e2c" exitCode=0
Jul 19 15:25:59.943540 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.943388    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb" event={"ID":"c31867ec-4612-4cb1-9321-10e892416814","Type":"ContainerDied","Data":"c439355a28bf815e54693f362431bde7a207f038b04b1023a77ee53b736c1e2c"}
Jul 19 15:25:59.964503 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.964445    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:25:59.995261 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995231    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-model-cache\") pod \"c31867ec-4612-4cb1-9321-10e892416814\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") "
Jul 19 15:25:59.995459 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995271    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-dshm\") pod \"c31867ec-4612-4cb1-9321-10e892416814\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") "
Jul 19 15:25:59.995459 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995296    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmpzn\" (UniqueName: \"kubernetes.io/projected/c31867ec-4612-4cb1-9321-10e892416814-kube-api-access-wmpzn\") pod \"c31867ec-4612-4cb1-9321-10e892416814\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") "
Jul 19 15:25:59.995459 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995372    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-tmp-dir\") pod \"c31867ec-4612-4cb1-9321-10e892416814\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") "
Jul 19 15:25:59.995459 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995408    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31867ec-4612-4cb1-9321-10e892416814-tls-certs\") pod \"c31867ec-4612-4cb1-9321-10e892416814\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") "
Jul 19 15:25:59.995459 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995436    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-home\") pod \"c31867ec-4612-4cb1-9321-10e892416814\" (UID: \"c31867ec-4612-4cb1-9321-10e892416814\") "
Jul 19 15:25:59.995730 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995690    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c31867ec-4612-4cb1-9321-10e892416814" (UID: "c31867ec-4612-4cb1-9321-10e892416814"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:59.995907 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995868    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-model-cache" (OuterVolumeSpecName: "model-cache") pod "c31867ec-4612-4cb1-9321-10e892416814" (UID: "c31867ec-4612-4cb1-9321-10e892416814"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:59.996036 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.995995    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-home" (OuterVolumeSpecName: "home") pod "c31867ec-4612-4cb1-9321-10e892416814" (UID: "c31867ec-4612-4cb1-9321-10e892416814"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:59.998137 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.998091    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c31867ec-4612-4cb1-9321-10e892416814-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c31867ec-4612-4cb1-9321-10e892416814" (UID: "c31867ec-4612-4cb1-9321-10e892416814"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:25:59.998245 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.998198    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-dshm" (OuterVolumeSpecName: "dshm") pod "c31867ec-4612-4cb1-9321-10e892416814" (UID: "c31867ec-4612-4cb1-9321-10e892416814"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:25:59.998326 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:25:59.998304    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c31867ec-4612-4cb1-9321-10e892416814-kube-api-access-wmpzn" (OuterVolumeSpecName: "kube-api-access-wmpzn") pod "c31867ec-4612-4cb1-9321-10e892416814" (UID: "c31867ec-4612-4cb1-9321-10e892416814"). InnerVolumeSpecName "kube-api-access-wmpzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:26:00.096049 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.096012    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmpzn\" (UniqueName: \"kubernetes.io/projected/c31867ec-4612-4cb1-9321-10e892416814-kube-api-access-wmpzn\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:26:00.096250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.096061    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:26:00.096250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.096072    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c31867ec-4612-4cb1-9321-10e892416814-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:26:00.096250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.096082    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:26:00.096250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.096089    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:26:00.096250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.096098    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c31867ec-4612-4cb1-9321-10e892416814-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:26:00.949997 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.949971    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"
Jul 19 15:26:00.950453 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.949959    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb" event={"ID":"c31867ec-4612-4cb1-9321-10e892416814","Type":"ContainerDied","Data":"560e4fd467c6fa643319f6f1c8be83f4142ee90f6a6f06a4f23c43f9919fcd23"}
Jul 19 15:26:00.950453 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.950103    2572 scope.go:117] "RemoveContainer" containerID="c439355a28bf815e54693f362431bde7a207f038b04b1023a77ee53b736c1e2c"
Jul 19 15:26:00.969403 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.969376    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"]
Jul 19 15:26:00.974805 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:00.974782    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-28015a4b/e2e-lifecycle-model-28015a4b-kserve-56f7fd9587-5r2nb"]
Jul 19 15:26:02.785757 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:02.785720    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c31867ec-4612-4cb1-9321-10e892416814" path="/var/lib/kubelet/pods/c31867ec-4612-4cb1-9321-10e892416814/volumes"
Jul 19 15:26:38.153895 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:38.153861    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:26:38.968303 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:38.968276    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:26:38.978741 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:38.978718    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:26:46.748520 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:26:46.748482    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:27:31.153188 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:27:31.153149    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:27:39.747035 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:27:39.746998    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:28:07.161712 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.161678    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"]
Jul 19 15:28:07.162202 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.162055    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c31867ec-4612-4cb1-9321-10e892416814" containerName="main"
Jul 19 15:28:07.162202 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.162067    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c31867ec-4612-4cb1-9321-10e892416814" containerName="main"
Jul 19 15:28:07.162202 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.162132    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c31867ec-4612-4cb1-9321-10e892416814" containerName="main"
Jul 19 15:28:07.165226 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.165207    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.169145 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.169114    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9d5834\"/\"default-dockercfg-hfxk4\""
Jul 19 15:28:07.169145 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.169142    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9d5834\"/\"kube-root-ca.crt\""
Jul 19 15:28:07.169395 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.169119    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-9d5834\"/\"openshift-service-ca.crt\""
Jul 19 15:28:07.169395 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.169114    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-9d5834\"/\"test-model-9d5834-kserve-self-signed-certs\""
Jul 19 15:28:07.177165 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.177135    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"]
Jul 19 15:28:07.341596 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.341555    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-home\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.341803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.341639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87qxl\" (UniqueName: \"kubernetes.io/projected/ca5da998-98a2-459a-b587-f9c92e265818-kube-api-access-87qxl\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.341803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.341680    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5da998-98a2-459a-b587-f9c92e265818-tls-certs\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.341803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.341698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-tmp-dir\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.341803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.341717    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-dshm\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.341803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.341791    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-model-cache\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.442521 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.442416    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87qxl\" (UniqueName: \"kubernetes.io/projected/ca5da998-98a2-459a-b587-f9c92e265818-kube-api-access-87qxl\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.442521 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.442470    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5da998-98a2-459a-b587-f9c92e265818-tls-certs\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.442768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.442643    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-tmp-dir\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.442768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.442677    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-dshm\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.442768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.442722    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-model-cache\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.442768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.442758    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-home\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.443062 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.443037    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-tmp-dir\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.443121 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.443071    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-model-cache\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.443208 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.443188    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-home\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.444996 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.444970    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-dshm\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.445220 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.445204    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5da998-98a2-459a-b587-f9c92e265818-tls-certs\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.450598 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.450579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87qxl\" (UniqueName: \"kubernetes.io/projected/ca5da998-98a2-459a-b587-f9c92e265818-kube-api-access-87qxl\") pod \"test-model-9d5834-kserve-568ccdc5d7-gnwtz\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") " pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.477724 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.477678    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:07.615735 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:07.615703    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"]
Jul 19 15:28:07.617278 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:28:07.617251    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca5da998_98a2_459a_b587_f9c92e265818.slice/crio-21ff8ec6f33bd6e3764b6cb69fe9d36d932beb61c8ebd263232568006a7cf42c WatchSource:0}: Error finding container 21ff8ec6f33bd6e3764b6cb69fe9d36d932beb61c8ebd263232568006a7cf42c: Status 404 returned error can't find the container with id 21ff8ec6f33bd6e3764b6cb69fe9d36d932beb61c8ebd263232568006a7cf42c
Jul 19 15:28:08.615192 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:08.615152    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz" event={"ID":"ca5da998-98a2-459a-b587-f9c92e265818","Type":"ContainerStarted","Data":"fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e"}
Jul 19 15:28:08.615192 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:08.615201    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz" event={"ID":"ca5da998-98a2-459a-b587-f9c92e265818","Type":"ContainerStarted","Data":"21ff8ec6f33bd6e3764b6cb69fe9d36d932beb61c8ebd263232568006a7cf42c"}
Jul 19 15:28:08.631708 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:08.631616    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz" podStartSLOduration=1.631595057 podStartE2EDuration="1.631595057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:28:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:28:08.630743722 +0000 UTC m=+2490.332191788" watchObservedRunningTime="2026-07-19 15:28:08.631595057 +0000 UTC m=+2490.333043109"
Jul 19 15:28:10.659744 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:10.659704    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:28:17.478636 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:17.478592    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:17.478636 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:17.478640    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:17.491445 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:17.491413    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:17.671947 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:17.671912    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:28:21.450695 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:21.450656    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:28:31.573549 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.573507    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"]
Jul 19 15:28:31.577154 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.577137    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.579810 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.579786    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-42a367\"/\"kube-root-ca.crt\""
Jul 19 15:28:31.581051 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.581020    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-42a367\"/\"openshift-service-ca.crt\""
Jul 19 15:28:31.581051 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.581021    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-42a367\"/\"test-model-42a367-kserve-self-signed-certs\""
Jul 19 15:28:31.581211 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.581074    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-42a367\"/\"default-dockercfg-jsmlw\""
Jul 19 15:28:31.585893 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.585872    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"]
Jul 19 15:28:31.652398 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.652361    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-tmp-dir\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.652642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.652417    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-home\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.652642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.652507    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-model-cache\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.652642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.652563    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2014b-a0ba-42f6-beb8-3400308879cd-tls-certs\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.652642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.652620    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69bxw\" (UniqueName: \"kubernetes.io/projected/e9e2014b-a0ba-42f6-beb8-3400308879cd-kube-api-access-69bxw\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.652820 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.652651    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-dshm\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754004    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-tmp-dir\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754059    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-home\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754264 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754097    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-model-cache\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754264 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754138    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2014b-a0ba-42f6-beb8-3400308879cd-tls-certs\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754376 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754278    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69bxw\" (UniqueName: \"kubernetes.io/projected/e9e2014b-a0ba-42f6-beb8-3400308879cd-kube-api-access-69bxw\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754376 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-dshm\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754579 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754555    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-model-cache\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754665 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-home\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.754665 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.754579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-tmp-dir\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.756574 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.756550    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-dshm\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.756887 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.756872    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2014b-a0ba-42f6-beb8-3400308879cd-tls-certs\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.762708 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.762689    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69bxw\" (UniqueName: \"kubernetes.io/projected/e9e2014b-a0ba-42f6-beb8-3400308879cd-kube-api-access-69bxw\") pod \"test-model-42a367-kserve-5d5c8dd48c-6vx6w\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") " pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:31.887893 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:31.887859    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:32.027068 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:32.027039    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"]
Jul 19 15:28:32.028995 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:28:32.028964    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9e2014b_a0ba_42f6_beb8_3400308879cd.slice/crio-f38f1dc87a9c3eb9fe59ccda0f27e89bc0c62941a083b66c91fd6b634df28e99 WatchSource:0}: Error finding container f38f1dc87a9c3eb9fe59ccda0f27e89bc0c62941a083b66c91fd6b634df28e99: Status 404 returned error can't find the container with id f38f1dc87a9c3eb9fe59ccda0f27e89bc0c62941a083b66c91fd6b634df28e99
Jul 19 15:28:32.736433 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:32.736387    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w" event={"ID":"e9e2014b-a0ba-42f6-beb8-3400308879cd","Type":"ContainerStarted","Data":"571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81"}
Jul 19 15:28:32.736857 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:32.736445    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w" event={"ID":"e9e2014b-a0ba-42f6-beb8-3400308879cd","Type":"ContainerStarted","Data":"f38f1dc87a9c3eb9fe59ccda0f27e89bc0c62941a083b66c91fd6b634df28e99"}
Jul 19 15:28:32.757044 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:32.756980    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w" podStartSLOduration=1.7569559890000002 podStartE2EDuration="1.756955989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:28:32.752619776 +0000 UTC m=+2514.454067828" watchObservedRunningTime="2026-07-19 15:28:32.756955989 +0000 UTC m=+2514.458404043"
Jul 19 15:28:36.963982 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:36.963948    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:28:41.888132 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:41.888087    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:41.888676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:41.888154    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:41.900641 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:41.900608    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:42.799469 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:42.799436    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:28:50.252225 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:50.252189    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:28:52.363472 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:52.363430    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"]
Jul 19 15:28:53.361553 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:28:53.361508    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"]
Jul 19 15:29:06.851601 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:06.851558    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:29:07.384375 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.384320    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz" podUID="ca5da998-98a2-459a-b587-f9c92e265818" containerName="main" containerID="cri-o://fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e" gracePeriod=45
Jul 19 15:29:07.641147 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.641077    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:29:07.675366 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675308    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-home\") pod \"ca5da998-98a2-459a-b587-f9c92e265818\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") "
Jul 19 15:29:07.675537 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675388    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-tmp-dir\") pod \"ca5da998-98a2-459a-b587-f9c92e265818\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") "
Jul 19 15:29:07.675537 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675482    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-model-cache\") pod \"ca5da998-98a2-459a-b587-f9c92e265818\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") "
Jul 19 15:29:07.675624 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675538    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5da998-98a2-459a-b587-f9c92e265818-tls-certs\") pod \"ca5da998-98a2-459a-b587-f9c92e265818\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") "
Jul 19 15:29:07.675624 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675568    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-dshm\") pod \"ca5da998-98a2-459a-b587-f9c92e265818\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") "
Jul 19 15:29:07.675624 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675598    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87qxl\" (UniqueName: \"kubernetes.io/projected/ca5da998-98a2-459a-b587-f9c92e265818-kube-api-access-87qxl\") pod \"ca5da998-98a2-459a-b587-f9c92e265818\" (UID: \"ca5da998-98a2-459a-b587-f9c92e265818\") "
Jul 19 15:29:07.675774 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675624    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca5da998-98a2-459a-b587-f9c92e265818" (UID: "ca5da998-98a2-459a-b587-f9c92e265818"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:07.675774 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675633    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-home" (OuterVolumeSpecName: "home") pod "ca5da998-98a2-459a-b587-f9c92e265818" (UID: "ca5da998-98a2-459a-b587-f9c92e265818"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:07.675774 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675757    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca5da998-98a2-459a-b587-f9c92e265818" (UID: "ca5da998-98a2-459a-b587-f9c92e265818"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:07.675920 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675902    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:07.675982 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675920    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:07.675982 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.675933    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:07.677973 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.677946    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca5da998-98a2-459a-b587-f9c92e265818-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca5da998-98a2-459a-b587-f9c92e265818" (UID: "ca5da998-98a2-459a-b587-f9c92e265818"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:29:07.677973 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.677952    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-dshm" (OuterVolumeSpecName: "dshm") pod "ca5da998-98a2-459a-b587-f9c92e265818" (UID: "ca5da998-98a2-459a-b587-f9c92e265818"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:07.678130 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.677999    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca5da998-98a2-459a-b587-f9c92e265818-kube-api-access-87qxl" (OuterVolumeSpecName: "kube-api-access-87qxl") pod "ca5da998-98a2-459a-b587-f9c92e265818" (UID: "ca5da998-98a2-459a-b587-f9c92e265818"). InnerVolumeSpecName "kube-api-access-87qxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:07.776703 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.776666    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5da998-98a2-459a-b587-f9c92e265818-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:07.776703 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.776698    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5da998-98a2-459a-b587-f9c92e265818-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:07.776703 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.776707    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87qxl\" (UniqueName: \"kubernetes.io/projected/ca5da998-98a2-459a-b587-f9c92e265818-kube-api-access-87qxl\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:07.923955 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.923857    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca5da998-98a2-459a-b587-f9c92e265818" containerID="fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e" exitCode=0
Jul 19 15:29:07.923955 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.923902    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"
Jul 19 15:29:07.923955 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.923936    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz" event={"ID":"ca5da998-98a2-459a-b587-f9c92e265818","Type":"ContainerDied","Data":"fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e"}
Jul 19 15:29:07.924524 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.923977    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz" event={"ID":"ca5da998-98a2-459a-b587-f9c92e265818","Type":"ContainerDied","Data":"21ff8ec6f33bd6e3764b6cb69fe9d36d932beb61c8ebd263232568006a7cf42c"}
Jul 19 15:29:07.924524 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.923995    2572 scope.go:117] "RemoveContainer" containerID="fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e"
Jul 19 15:29:07.933754 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.933734    2572 scope.go:117] "RemoveContainer" containerID="fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e"
Jul 19 15:29:07.934059 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:29:07.934041    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e\": container with ID starting with fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e not found: ID does not exist" containerID="fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e"
Jul 19 15:29:07.934105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.934070    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e"} err="failed to get container status \"fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e\": rpc error: code = NotFound desc = could not find container \"fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e\": container with ID starting with fc40ef0ad781329add0370a5ca8ed99a6d9b9b67b47d30613d76613f8c15698e not found: ID does not exist"
Jul 19 15:29:07.945043 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.945017    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"]
Jul 19 15:29:07.948732 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:07.948708    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-9d5834/test-model-9d5834-kserve-568ccdc5d7-gnwtz"]
Jul 19 15:29:08.382642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.382600    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w" podUID="e9e2014b-a0ba-42f6-beb8-3400308879cd" containerName="main" containerID="cri-o://571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81" gracePeriod=45
Jul 19 15:29:08.631294 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.631273    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:29:08.684643 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684552    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-dshm\") pod \"e9e2014b-a0ba-42f6-beb8-3400308879cd\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") "
Jul 19 15:29:08.684643 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684597    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-home\") pod \"e9e2014b-a0ba-42f6-beb8-3400308879cd\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") "
Jul 19 15:29:08.684643 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684629    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-model-cache\") pod \"e9e2014b-a0ba-42f6-beb8-3400308879cd\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") "
Jul 19 15:29:08.684918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684663    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69bxw\" (UniqueName: \"kubernetes.io/projected/e9e2014b-a0ba-42f6-beb8-3400308879cd-kube-api-access-69bxw\") pod \"e9e2014b-a0ba-42f6-beb8-3400308879cd\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") "
Jul 19 15:29:08.684918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684691    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2014b-a0ba-42f6-beb8-3400308879cd-tls-certs\") pod \"e9e2014b-a0ba-42f6-beb8-3400308879cd\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") "
Jul 19 15:29:08.684918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684758    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-tmp-dir\") pod \"e9e2014b-a0ba-42f6-beb8-3400308879cd\" (UID: \"e9e2014b-a0ba-42f6-beb8-3400308879cd\") "
Jul 19 15:29:08.684918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684844    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-home" (OuterVolumeSpecName: "home") pod "e9e2014b-a0ba-42f6-beb8-3400308879cd" (UID: "e9e2014b-a0ba-42f6-beb8-3400308879cd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:08.684918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.684877    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9e2014b-a0ba-42f6-beb8-3400308879cd" (UID: "e9e2014b-a0ba-42f6-beb8-3400308879cd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:08.685145 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.685054    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9e2014b-a0ba-42f6-beb8-3400308879cd" (UID: "e9e2014b-a0ba-42f6-beb8-3400308879cd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:08.685145 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.685092    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:08.685145 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.685111    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:08.687019 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.686996    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-dshm" (OuterVolumeSpecName: "dshm") pod "e9e2014b-a0ba-42f6-beb8-3400308879cd" (UID: "e9e2014b-a0ba-42f6-beb8-3400308879cd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:29:08.687019 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.687010    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e2014b-a0ba-42f6-beb8-3400308879cd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9e2014b-a0ba-42f6-beb8-3400308879cd" (UID: "e9e2014b-a0ba-42f6-beb8-3400308879cd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:29:08.687121 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.687011    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e2014b-a0ba-42f6-beb8-3400308879cd-kube-api-access-69bxw" (OuterVolumeSpecName: "kube-api-access-69bxw") pod "e9e2014b-a0ba-42f6-beb8-3400308879cd" (UID: "e9e2014b-a0ba-42f6-beb8-3400308879cd"). InnerVolumeSpecName "kube-api-access-69bxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:29:08.786142 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.786114    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:08.786142 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.786138    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2014b-a0ba-42f6-beb8-3400308879cd-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:08.786142 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.786147    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69bxw\" (UniqueName: \"kubernetes.io/projected/e9e2014b-a0ba-42f6-beb8-3400308879cd-kube-api-access-69bxw\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:08.786402 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.786157    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2014b-a0ba-42f6-beb8-3400308879cd-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:08.786970 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.786945    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca5da998-98a2-459a-b587-f9c92e265818" path="/var/lib/kubelet/pods/ca5da998-98a2-459a-b587-f9c92e265818/volumes"
Jul 19 15:29:08.930584 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.930541    2572 generic.go:356] "Generic (PLEG): container finished" podID="e9e2014b-a0ba-42f6-beb8-3400308879cd" containerID="571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81" exitCode=0
Jul 19 15:29:08.930989 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.930591    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w" event={"ID":"e9e2014b-a0ba-42f6-beb8-3400308879cd","Type":"ContainerDied","Data":"571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81"}
Jul 19 15:29:08.930989 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.930610    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"
Jul 19 15:29:08.930989 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.930635    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w" event={"ID":"e9e2014b-a0ba-42f6-beb8-3400308879cd","Type":"ContainerDied","Data":"f38f1dc87a9c3eb9fe59ccda0f27e89bc0c62941a083b66c91fd6b634df28e99"}
Jul 19 15:29:08.930989 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.930655    2572 scope.go:117] "RemoveContainer" containerID="571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81"
Jul 19 15:29:08.940048 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.940027    2572 scope.go:117] "RemoveContainer" containerID="571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81"
Jul 19 15:29:08.940359 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:29:08.940323    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81\": container with ID starting with 571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81 not found: ID does not exist" containerID="571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81"
Jul 19 15:29:08.940431 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.940370    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81"} err="failed to get container status \"571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81\": rpc error: code = NotFound desc = could not find container \"571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81\": container with ID starting with 571c92fc045f3fb8e1181487e2d76f2df7a606c669c982b898f3799e14aeee81 not found: ID does not exist"
Jul 19 15:29:08.947272 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.947243    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"]
Jul 19 15:29:08.951995 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:08.951968    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-42a367/test-model-42a367-kserve-5d5c8dd48c-6vx6w"]
Jul 19 15:29:10.785738 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:10.785708    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e2014b-a0ba-42f6-beb8-3400308879cd" path="/var/lib/kubelet/pods/e9e2014b-a0ba-42f6-beb8-3400308879cd/volumes"
Jul 19 15:29:15.347193 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347157    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"]
Jul 19 15:29:15.347721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347554    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca5da998-98a2-459a-b587-f9c92e265818" containerName="main"
Jul 19 15:29:15.347721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347566    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca5da998-98a2-459a-b587-f9c92e265818" containerName="main"
Jul 19 15:29:15.347721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347584    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e2014b-a0ba-42f6-beb8-3400308879cd" containerName="main"
Jul 19 15:29:15.347721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347589    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e2014b-a0ba-42f6-beb8-3400308879cd" containerName="main"
Jul 19 15:29:15.347721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347643    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca5da998-98a2-459a-b587-f9c92e265818" containerName="main"
Jul 19 15:29:15.347721 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.347651    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e2014b-a0ba-42f6-beb8-3400308879cd" containerName="main"
Jul 19 15:29:15.350530 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.350514    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:15.353175 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.353152    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 15:29:15.353301 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.353153    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:29:15.357734 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.357706    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"]
Jul 19 15:29:15.443923 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.443885    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg\" (UID: \"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:15.544768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.544727    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg\" (UID: \"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:15.545360 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.545313    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg\" (UID: \"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:15.661598 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.661505    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:15.798402 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.798374    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"]
Jul 19 15:29:15.799936 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:29:15.799906    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723dc1b3_dfc3_4c60_aaa6_e9b021a05c8d.slice/crio-f066e3c7be11f819e387328b2c7e7c1edb22b2518cdfde001d3a157a7bcd006f WatchSource:0}: Error finding container f066e3c7be11f819e387328b2c7e7c1edb22b2518cdfde001d3a157a7bcd006f: Status 404 returned error can't find the container with id f066e3c7be11f819e387328b2c7e7c1edb22b2518cdfde001d3a157a7bcd006f
Jul 19 15:29:15.801840 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.801826    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:29:15.975152 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:15.975068    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg" event={"ID":"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d","Type":"ContainerStarted","Data":"f066e3c7be11f819e387328b2c7e7c1edb22b2518cdfde001d3a157a7bcd006f"}
Jul 19 15:29:17.988138 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:17.988109    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg" event={"ID":"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d","Type":"ContainerStarted","Data":"66d57724d1a91d0d9cad4d4afe657854cebdf0df23ce318108fd5ebbb56f4ae9"}
Jul 19 15:29:18.006291 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:18.006237    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg" podStartSLOduration=1.103477944 podStartE2EDuration="3.006220029s" totalImagesPullingTime="1.902742085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:29:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:29:18.004178744 +0000 UTC m=+2559.705626797" watchObservedRunningTime="2026-07-19 15:29:18.006220029 +0000 UTC m=+2559.707668081"
Jul 19 15:29:19.000846 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:19.000806    2572 generic.go:356] "Generic (PLEG): container finished" podID="723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d" containerID="66d57724d1a91d0d9cad4d4afe657854cebdf0df23ce318108fd5ebbb56f4ae9" exitCode=0
Jul 19 15:29:19.001221 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:19.000892    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg" event={"ID":"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d","Type":"ContainerDied","Data":"66d57724d1a91d0d9cad4d4afe657854cebdf0df23ce318108fd5ebbb56f4ae9"}
Jul 19 15:29:20.132911 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:20.132887    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:20.185676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:20.185639    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d-maas-api-service-ca\") pod \"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d\" (UID: \"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d\") "
Jul 19 15:29:20.185994 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:20.185973    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d" (UID: "723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:29:20.286797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:20.286710    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d-maas-api-service-ca\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:29:21.012784 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:21.012755    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"
Jul 19 15:29:21.012784 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:21.012768    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg" event={"ID":"723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d","Type":"ContainerDied","Data":"f066e3c7be11f819e387328b2c7e7c1edb22b2518cdfde001d3a157a7bcd006f"}
Jul 19 15:29:21.012984 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:21.012813    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f066e3c7be11f819e387328b2c7e7c1edb22b2518cdfde001d3a157a7bcd006f"
Jul 19 15:29:28.767644 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:28.767608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:29:30.427275 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:30.427238    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"]
Jul 19 15:29:30.429252 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:30.429227    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-42a367-dxsmg"]
Jul 19 15:29:30.785561 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:30.785488    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d" path="/var/lib/kubelet/pods/723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d/volumes"
Jul 19 15:29:43.454123 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:43.454085    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:29:55.759624 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:29:55.759582    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:30:00.138459 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.138375    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"]
Jul 19 15:30:00.138830 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.138812    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d" containerName="revoke-keys"
Jul 19 15:30:00.138870 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.138833    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d" containerName="revoke-keys"
Jul 19 15:30:00.138910 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.138903    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="723dc1b3-dfc3-4c60-aaa6-e9b021a05c8d" containerName="revoke-keys"
Jul 19 15:30:00.149759 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.149725    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"
Jul 19 15:30:00.152662 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.152577    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:30:00.152899 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.152876    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"]
Jul 19 15:30:00.156753 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.156727    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"]
Jul 19 15:30:00.157981 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.157494    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"
Jul 19 15:30:00.161029 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.160999    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"]
Jul 19 15:30:00.161151 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.161042    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"]
Jul 19 15:30:00.161219 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.161154    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"
Jul 19 15:30:00.162946 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.162920    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"]
Jul 19 15:30:00.164966 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.164948    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"
Jul 19 15:30:00.173260 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.173237    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"
Jul 19 15:30:00.208178 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.208149    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"
Jul 19 15:30:00.332037 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.332003    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"]
Jul 19 15:30:00.334153 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:30:00.334098    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod077f3c67_4e55_4ee9_a270_7b404de6582a.slice/crio-6001f1af6c9e7361d268ef39131f5e6fd4d4e5101cfdae07db7939fc93a6f959 WatchSource:0}: Error finding container 6001f1af6c9e7361d268ef39131f5e6fd4d4e5101cfdae07db7939fc93a6f959: Status 404 returned error can't find the container with id 6001f1af6c9e7361d268ef39131f5e6fd4d4e5101cfdae07db7939fc93a6f959
Jul 19 15:30:00.363517 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.363479    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"]
Jul 19 15:30:00.363894 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:30:00.363865    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02bf2ce4_ae27_4472_ba7d_f3a4e00d44e3.slice/crio-e05caed09efb08d22cfcea5d1c7d85f41b180a80bad749d12da76f5d503acd01 WatchSource:0}: Error finding container e05caed09efb08d22cfcea5d1c7d85f41b180a80bad749d12da76f5d503acd01: Status 404 returned error can't find the container with id e05caed09efb08d22cfcea5d1c7d85f41b180a80bad749d12da76f5d503acd01
Jul 19 15:30:00.388913 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:00.388887    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"]
Jul 19 15:30:00.392594 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:30:00.392569    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a602c5_b973_4b2e_a776_5b49616f216b.slice/crio-4e7645b1b72bbe5a3681d31dd68c6f627213244f7000ab76d30128047776a85f WatchSource:0}: Error finding container 4e7645b1b72bbe5a3681d31dd68c6f627213244f7000ab76d30128047776a85f: Status 404 returned error can't find the container with id 4e7645b1b72bbe5a3681d31dd68c6f627213244f7000ab76d30128047776a85f
Jul 19 15:30:01.134354 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.134299    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"]
Jul 19 15:30:01.137611 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.137594    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:01.140042 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.140024    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 15:30:01.145253 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.145232    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"]
Jul 19 15:30:01.155693 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.155662    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/347bb2d9-cf6a-417b-88ac-000abc986faf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-20b742-ms45w\" (UID: \"347bb2d9-cf6a-417b-88ac-000abc986faf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:01.227075 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.227028    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c" event={"ID":"02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3","Type":"ContainerStarted","Data":"68748bc96827fb296cf0d1c1d38e1d3c8b0139ecaa7feaaea020dd0de6977519"}
Jul 19 15:30:01.227274 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.227095    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c" event={"ID":"02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3","Type":"ContainerStarted","Data":"e05caed09efb08d22cfcea5d1c7d85f41b180a80bad749d12da76f5d503acd01"}
Jul 19 15:30:01.228819 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.228783    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr" event={"ID":"b7a602c5-b973-4b2e-a776-5b49616f216b","Type":"ContainerStarted","Data":"502cbf0ba595a1bab411eff7434235e59097f104593b7a8463e581760a5f6ecc"}
Jul 19 15:30:01.228944 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.228833    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr" event={"ID":"b7a602c5-b973-4b2e-a776-5b49616f216b","Type":"ContainerStarted","Data":"4e7645b1b72bbe5a3681d31dd68c6f627213244f7000ab76d30128047776a85f"}
Jul 19 15:30:01.230249 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.230224    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv" event={"ID":"077f3c67-4e55-4ee9-a270-7b404de6582a","Type":"ContainerStarted","Data":"1a7c6fb865e367d4a8e7ec5100b219afc27b011781e7513abfad6cca3b2ccf26"}
Jul 19 15:30:01.230398 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.230266    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv" event={"ID":"077f3c67-4e55-4ee9-a270-7b404de6582a","Type":"ContainerStarted","Data":"6001f1af6c9e7361d268ef39131f5e6fd4d4e5101cfdae07db7939fc93a6f959"}
Jul 19 15:30:01.244035 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.243983    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c" podStartSLOduration=1.243966436 podStartE2EDuration="1.243966436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:30:01.24249547 +0000 UTC m=+2602.943943534" watchObservedRunningTime="2026-07-19 15:30:01.243966436 +0000 UTC m=+2602.945414468"
Jul 19 15:30:01.256873 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.256838    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/347bb2d9-cf6a-417b-88ac-000abc986faf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-20b742-ms45w\" (UID: \"347bb2d9-cf6a-417b-88ac-000abc986faf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:01.257516 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.257500    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/347bb2d9-cf6a-417b-88ac-000abc986faf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-20b742-ms45w\" (UID: \"347bb2d9-cf6a-417b-88ac-000abc986faf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:01.261212 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.261168    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr" podStartSLOduration=1.261153626 podStartE2EDuration="1.261153626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:30:01.260063722 +0000 UTC m=+2602.961511774" watchObservedRunningTime="2026-07-19 15:30:01.261153626 +0000 UTC m=+2602.962601740"
Jul 19 15:30:01.278778 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.278717    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv" podStartSLOduration=1.278691521 podStartE2EDuration="1.278691521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:30:01.276682462 +0000 UTC m=+2602.978130527" watchObservedRunningTime="2026-07-19 15:30:01.278691521 +0000 UTC m=+2602.980139577"
Jul 19 15:30:01.449798 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.449690    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:01.590509 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:01.590484    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"]
Jul 19 15:30:01.684300 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:30:01.684268    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347bb2d9_cf6a_417b_88ac_000abc986faf.slice/crio-43dd667557c64875ccc0ce724139c75b2880af077a599b4dba744fe1dff73593 WatchSource:0}: Error finding container 43dd667557c64875ccc0ce724139c75b2880af077a599b4dba744fe1dff73593: Status 404 returned error can't find the container with id 43dd667557c64875ccc0ce724139c75b2880af077a599b4dba744fe1dff73593
Jul 19 15:30:02.237663 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.237628    2572 generic.go:356] "Generic (PLEG): container finished" podID="077f3c67-4e55-4ee9-a270-7b404de6582a" containerID="1a7c6fb865e367d4a8e7ec5100b219afc27b011781e7513abfad6cca3b2ccf26" exitCode=0
Jul 19 15:30:02.238133 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.237674    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv" event={"ID":"077f3c67-4e55-4ee9-a270-7b404de6582a","Type":"ContainerDied","Data":"1a7c6fb865e367d4a8e7ec5100b219afc27b011781e7513abfad6cca3b2ccf26"}
Jul 19 15:30:02.239374 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.239326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w" event={"ID":"347bb2d9-cf6a-417b-88ac-000abc986faf","Type":"ContainerStarted","Data":"abd401794eb8f12072b09321e6a9c121ccc0f22e1529a71b047e2b30cfb06038"}
Jul 19 15:30:02.239519 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.239393    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w" event={"ID":"347bb2d9-cf6a-417b-88ac-000abc986faf","Type":"ContainerStarted","Data":"43dd667557c64875ccc0ce724139c75b2880af077a599b4dba744fe1dff73593"}
Jul 19 15:30:02.240857 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.240836    2572 generic.go:356] "Generic (PLEG): container finished" podID="02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3" containerID="68748bc96827fb296cf0d1c1d38e1d3c8b0139ecaa7feaaea020dd0de6977519" exitCode=0
Jul 19 15:30:02.240969 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.240914    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c" event={"ID":"02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3","Type":"ContainerDied","Data":"68748bc96827fb296cf0d1c1d38e1d3c8b0139ecaa7feaaea020dd0de6977519"}
Jul 19 15:30:02.242443 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.242420    2572 generic.go:356] "Generic (PLEG): container finished" podID="b7a602c5-b973-4b2e-a776-5b49616f216b" containerID="502cbf0ba595a1bab411eff7434235e59097f104593b7a8463e581760a5f6ecc" exitCode=0
Jul 19 15:30:02.242528 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.242491    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr" event={"ID":"b7a602c5-b973-4b2e-a776-5b49616f216b","Type":"ContainerDied","Data":"502cbf0ba595a1bab411eff7434235e59097f104593b7a8463e581760a5f6ecc"}
Jul 19 15:30:02.266933 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:02.266671    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w" podStartSLOduration=1.266653051 podStartE2EDuration="1.266653051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:30:02.266195837 +0000 UTC m=+2603.967643890" watchObservedRunningTime="2026-07-19 15:30:02.266653051 +0000 UTC m=+2603.968101107"
Jul 19 15:30:03.249199 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:03.249158    2572 generic.go:356] "Generic (PLEG): container finished" podID="347bb2d9-cf6a-417b-88ac-000abc986faf" containerID="abd401794eb8f12072b09321e6a9c121ccc0f22e1529a71b047e2b30cfb06038" exitCode=0
Jul 19 15:30:03.249694 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:03.249246    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w" event={"ID":"347bb2d9-cf6a-417b-88ac-000abc986faf","Type":"ContainerDied","Data":"abd401794eb8f12072b09321e6a9c121ccc0f22e1529a71b047e2b30cfb06038"}
Jul 19 15:30:03.408657 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:03.408630    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"
Jul 19 15:30:03.453486 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:03.453462    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"
Jul 19 15:30:03.457861 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:03.457833    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"
Jul 19 15:30:04.255601 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.255568    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv"
Jul 19 15:30:04.256056 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.255601    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29741250-zt5gv" event={"ID":"077f3c67-4e55-4ee9-a270-7b404de6582a","Type":"ContainerDied","Data":"6001f1af6c9e7361d268ef39131f5e6fd4d4e5101cfdae07db7939fc93a6f959"}
Jul 19 15:30:04.256056 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.255661    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6001f1af6c9e7361d268ef39131f5e6fd4d4e5101cfdae07db7939fc93a6f959"
Jul 19 15:30:04.257029 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.257010    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"
Jul 19 15:30:04.257029 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.257027    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c" event={"ID":"02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3","Type":"ContainerDied","Data":"e05caed09efb08d22cfcea5d1c7d85f41b180a80bad749d12da76f5d503acd01"}
Jul 19 15:30:04.257194 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.257060    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e05caed09efb08d22cfcea5d1c7d85f41b180a80bad749d12da76f5d503acd01"
Jul 19 15:30:04.258373 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.258328    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr" event={"ID":"b7a602c5-b973-4b2e-a776-5b49616f216b","Type":"ContainerDied","Data":"4e7645b1b72bbe5a3681d31dd68c6f627213244f7000ab76d30128047776a85f"}
Jul 19 15:30:04.258462 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.258381    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"
Jul 19 15:30:04.258462 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.258391    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e7645b1b72bbe5a3681d31dd68c6f627213244f7000ab76d30128047776a85f"
Jul 19 15:30:04.381803 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.381776    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:04.486715 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.486681    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/347bb2d9-cf6a-417b-88ac-000abc986faf-maas-api-service-ca\") pod \"347bb2d9-cf6a-417b-88ac-000abc986faf\" (UID: \"347bb2d9-cf6a-417b-88ac-000abc986faf\") "
Jul 19 15:30:04.487022 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.487000    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/347bb2d9-cf6a-417b-88ac-000abc986faf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "347bb2d9-cf6a-417b-88ac-000abc986faf" (UID: "347bb2d9-cf6a-417b-88ac-000abc986faf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:30:04.588215 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:04.588184    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/347bb2d9-cf6a-417b-88ac-000abc986faf-maas-api-service-ca\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:30:05.265228 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:05.265199    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"
Jul 19 15:30:05.265641 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:05.265226    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w" event={"ID":"347bb2d9-cf6a-417b-88ac-000abc986faf","Type":"ContainerDied","Data":"43dd667557c64875ccc0ce724139c75b2880af077a599b4dba744fe1dff73593"}
Jul 19 15:30:05.265641 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:05.265272    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="43dd667557c64875ccc0ce724139c75b2880af077a599b4dba744fe1dff73593"
Jul 19 15:30:17.042983 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:17.042943    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"]
Jul 19 15:30:17.049518 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:17.049476    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-20b742-29741250-k6hjr"]
Jul 19 15:30:18.785922 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:18.785887    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7a602c5-b973-4b2e-a776-5b49616f216b" path="/var/lib/kubelet/pods/b7a602c5-b973-4b2e-a776-5b49616f216b/volumes"
Jul 19 15:30:21.206060 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:21.206019    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"]
Jul 19 15:30:21.208514 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:21.208486    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-20b742-ms45w"]
Jul 19 15:30:22.755309 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:22.755278    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:30:22.785217 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:22.785185    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="347bb2d9-cf6a-417b-88ac-000abc986faf" path="/var/lib/kubelet/pods/347bb2d9-cf6a-417b-88ac-000abc986faf/volumes"
Jul 19 15:30:27.991047 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991009    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"]
Jul 19 15:30:27.991482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991421    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3" containerName="cleanup"
Jul 19 15:30:27.991482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991433    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3" containerName="cleanup"
Jul 19 15:30:27.991482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991446    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="077f3c67-4e55-4ee9-a270-7b404de6582a" containerName="cleanup"
Jul 19 15:30:27.991482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991454    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="077f3c67-4e55-4ee9-a270-7b404de6582a" containerName="cleanup"
Jul 19 15:30:27.991482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991469    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a602c5-b973-4b2e-a776-5b49616f216b" containerName="cleanup"
Jul 19 15:30:27.991482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991477    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a602c5-b973-4b2e-a776-5b49616f216b" containerName="cleanup"
Jul 19 15:30:27.991671 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991487    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="347bb2d9-cf6a-417b-88ac-000abc986faf" containerName="revoke-keys"
Jul 19 15:30:27.991671 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991493    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="347bb2d9-cf6a-417b-88ac-000abc986faf" containerName="revoke-keys"
Jul 19 15:30:27.991671 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991555    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="077f3c67-4e55-4ee9-a270-7b404de6582a" containerName="cleanup"
Jul 19 15:30:27.991671 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991563    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="347bb2d9-cf6a-417b-88ac-000abc986faf" containerName="revoke-keys"
Jul 19 15:30:27.991671 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991569    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3" containerName="cleanup"
Jul 19 15:30:27.991671 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.991574    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a602c5-b973-4b2e-a776-5b49616f216b" containerName="cleanup"
Jul 19 15:30:27.994806 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.994789    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:27.997428 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.997403    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 15:30:27.997553 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:27.997416    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:30:28.003969 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:28.003945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"]
Jul 19 15:30:28.100658 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:28.100619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7354720-1fd8-4f46-bfdf-1af359066f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-accabb-464np\" (UID: \"b7354720-1fd8-4f46-bfdf-1af359066f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:28.201346 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:28.201309    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7354720-1fd8-4f46-bfdf-1af359066f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-accabb-464np\" (UID: \"b7354720-1fd8-4f46-bfdf-1af359066f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:28.201993 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:28.201972    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7354720-1fd8-4f46-bfdf-1af359066f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-accabb-464np\" (UID: \"b7354720-1fd8-4f46-bfdf-1af359066f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:28.306612 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:28.306511    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:28.443767 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:28.443738    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"]
Jul 19 15:30:28.445967 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:30:28.445929    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7354720_1fd8_4f46_bfdf_1af359066f9f.slice/crio-20ae25b6b9f88afb137d976511eb4361aefbd07f18153787816948a259a4f8d9 WatchSource:0}: Error finding container 20ae25b6b9f88afb137d976511eb4361aefbd07f18153787816948a259a4f8d9: Status 404 returned error can't find the container with id 20ae25b6b9f88afb137d976511eb4361aefbd07f18153787816948a259a4f8d9
Jul 19 15:30:29.399985 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:29.399948    2572 generic.go:356] "Generic (PLEG): container finished" podID="b7354720-1fd8-4f46-bfdf-1af359066f9f" containerID="8451446f816af3158ddce61a44293942ab80df91eaed45be5187653f27b6cd0a" exitCode=0
Jul 19 15:30:29.400428 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:29.400042    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np" event={"ID":"b7354720-1fd8-4f46-bfdf-1af359066f9f","Type":"ContainerDied","Data":"8451446f816af3158ddce61a44293942ab80df91eaed45be5187653f27b6cd0a"}
Jul 19 15:30:29.400428 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:29.400144    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np" event={"ID":"b7354720-1fd8-4f46-bfdf-1af359066f9f","Type":"ContainerStarted","Data":"20ae25b6b9f88afb137d976511eb4361aefbd07f18153787816948a259a4f8d9"}
Jul 19 15:30:30.543452 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:30.543422    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:30.726905 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:30.726798    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7354720-1fd8-4f46-bfdf-1af359066f9f-maas-api-service-ca\") pod \"b7354720-1fd8-4f46-bfdf-1af359066f9f\" (UID: \"b7354720-1fd8-4f46-bfdf-1af359066f9f\") "
Jul 19 15:30:30.727214 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:30.727191    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7354720-1fd8-4f46-bfdf-1af359066f9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b7354720-1fd8-4f46-bfdf-1af359066f9f" (UID: "b7354720-1fd8-4f46-bfdf-1af359066f9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:30:30.828412 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:30.828381    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b7354720-1fd8-4f46-bfdf-1af359066f9f-maas-api-service-ca\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:30:31.412003 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:31.411974    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"
Jul 19 15:30:31.412188 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:31.412003    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np" event={"ID":"b7354720-1fd8-4f46-bfdf-1af359066f9f","Type":"ContainerDied","Data":"20ae25b6b9f88afb137d976511eb4361aefbd07f18153787816948a259a4f8d9"}
Jul 19 15:30:31.412188 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:31.412051    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20ae25b6b9f88afb137d976511eb4361aefbd07f18153787816948a259a4f8d9"
Jul 19 15:30:43.479408 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:43.479371    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"]
Jul 19 15:30:43.481652 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:43.481624    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-accabb-29741250-rwh8c"]
Jul 19 15:30:44.785639 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:44.785604    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3" path="/var/lib/kubelet/pods/02bf2ce4-ae27-4472-ba7d-f3a4e00d44e3/volumes"
Jul 19 15:30:47.993601 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:47.993562    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:30:48.064580 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:48.064537    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"]
Jul 19 15:30:48.067141 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:48.067119    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-accabb-464np"]
Jul 19 15:30:48.787197 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:30:48.787151    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7354720-1fd8-4f46-bfdf-1af359066f9f" path="/var/lib/kubelet/pods/b7354720-1fd8-4f46-bfdf-1af359066f9f/volumes"
Jul 19 15:31:05.659853 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:05.659816    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:31:16.673229 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:16.673177    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:31:28.456924 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.456887    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"]
Jul 19 15:31:28.457477 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.457415    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7354720-1fd8-4f46-bfdf-1af359066f9f" containerName="revoke-keys"
Jul 19 15:31:28.457477 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.457433    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7354720-1fd8-4f46-bfdf-1af359066f9f" containerName="revoke-keys"
Jul 19 15:31:28.457560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.457511    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7354720-1fd8-4f46-bfdf-1af359066f9f" containerName="revoke-keys"
Jul 19 15:31:28.460664 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.460646    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.463452 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.463423    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"kube-root-ca.crt\""
Jul 19 15:31:28.464506 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.464490    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"auth-test-model-4bc178-kserve-self-signed-certs\""
Jul 19 15:31:28.464594 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.464519    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"default-dockercfg-clv5l\""
Jul 19 15:31:28.464594 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.464527    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"openshift-service-ca.crt\""
Jul 19 15:31:28.471911 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.471884    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"]
Jul 19 15:31:28.539199 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.539156    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-dshm\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.539430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.539213    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14e69590-8061-4c6c-a71b-ff443a158a24-tls-certs\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.539430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.539241    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-model-cache\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.539430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.539263    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-tmp-dir\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.539430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.539281    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7lwq\" (UniqueName: \"kubernetes.io/projected/14e69590-8061-4c6c-a71b-ff443a158a24-kube-api-access-w7lwq\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.539430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.539417    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-home\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.640503 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640442    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-dshm\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.640720 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640535    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14e69590-8061-4c6c-a71b-ff443a158a24-tls-certs\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.640720 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640558    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-model-cache\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.640720 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640582    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-tmp-dir\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.640720 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7lwq\" (UniqueName: \"kubernetes.io/projected/14e69590-8061-4c6c-a71b-ff443a158a24-kube-api-access-w7lwq\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.640961 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640729    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-home\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.641021 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.640995    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-model-cache\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.641077 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.641029    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-tmp-dir\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.641130 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.641078    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-home\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.643100 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.643072    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-dshm\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.643233 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.643219    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14e69590-8061-4c6c-a71b-ff443a158a24-tls-certs\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.649652 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.649629    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7lwq\" (UniqueName: \"kubernetes.io/projected/14e69590-8061-4c6c-a71b-ff443a158a24-kube-api-access-w7lwq\") pod \"auth-test-model-4bc178-kserve-865f674d59-l85hx\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") " pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.770520 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.770424    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:28.924430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:28.924393    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"]
Jul 19 15:31:28.926293 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:31:28.926263    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14e69590_8061_4c6c_a71b_ff443a158a24.slice/crio-8400e70a04016cb6c2916e9920e4c46a40f2198b05eb84f2995198a9a70d6993 WatchSource:0}: Error finding container 8400e70a04016cb6c2916e9920e4c46a40f2198b05eb84f2995198a9a70d6993: Status 404 returned error can't find the container with id 8400e70a04016cb6c2916e9920e4c46a40f2198b05eb84f2995198a9a70d6993
Jul 19 15:31:29.741720 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:29.741614    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx" event={"ID":"14e69590-8061-4c6c-a71b-ff443a158a24","Type":"ContainerStarted","Data":"4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea"}
Jul 19 15:31:29.741720 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:29.741666    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx" event={"ID":"14e69590-8061-4c6c-a71b-ff443a158a24","Type":"ContainerStarted","Data":"8400e70a04016cb6c2916e9920e4c46a40f2198b05eb84f2995198a9a70d6993"}
Jul 19 15:31:29.759146 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:29.759091    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx" podStartSLOduration=1.759075114 podStartE2EDuration="1.759075114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:31:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:31:29.75665187 +0000 UTC m=+2691.458099928" watchObservedRunningTime="2026-07-19 15:31:29.759075114 +0000 UTC m=+2691.460523167"
Jul 19 15:31:32.053644 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:32.053608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:31:38.771067 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:38.771027    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:38.771503 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:38.771085    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:38.787597 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:38.787563    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:38.806101 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:38.806074    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:31:39.004169 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:39.004136    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:31:39.016380 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:39.016350    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:31:52.773149 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.773115    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"]
Jul 19 15:31:52.777161 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.777141    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.780934 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.780910    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"kube-root-ca.crt\""
Jul 19 15:31:52.781076 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.780945    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"openshift-service-ca.crt\""
Jul 19 15:31:52.781076 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.780911    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"auth-test-model-ccd7ab-kserve-self-signed-certs\""
Jul 19 15:31:52.781076 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.780962    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"default-dockercfg-sq9bp\""
Jul 19 15:31:52.788437 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.788414    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"]
Jul 19 15:31:52.858109 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.858074    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlh7b\" (UniqueName: \"kubernetes.io/projected/a4ec66e2-085b-4afa-92c8-2397941a882d-kube-api-access-jlh7b\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.858316 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.858115    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-model-cache\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.858316 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.858148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-dshm\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.858316 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.858254    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-tmp-dir\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.858485 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.858313    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-home\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.858485 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.858422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ec66e2-085b-4afa-92c8-2397941a882d-tls-certs\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959293 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959246    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-dshm\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959293 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959304    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-tmp-dir\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959605 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959370    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-home\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959605 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ec66e2-085b-4afa-92c8-2397941a882d-tls-certs\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959605 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959448    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlh7b\" (UniqueName: \"kubernetes.io/projected/a4ec66e2-085b-4afa-92c8-2397941a882d-kube-api-access-jlh7b\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959605 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959517    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-model-cache\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959851 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959823    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-tmp-dir\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959910 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959837    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-home\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.959910 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.959894    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-model-cache\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.961901 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.961877    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-dshm\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.962049 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.962032    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ec66e2-085b-4afa-92c8-2397941a882d-tls-certs\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:52.972699 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:52.972669    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlh7b\" (UniqueName: \"kubernetes.io/projected/a4ec66e2-085b-4afa-92c8-2397941a882d-kube-api-access-jlh7b\") pod \"auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") " pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:53.088085 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:53.088047    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:31:53.229864 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:53.229835    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"]
Jul 19 15:31:53.231277 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:31:53.231255    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4ec66e2_085b_4afa_92c8_2397941a882d.slice/crio-d5c7bfe522e646c06d7d7f35d8da18325cd495ff10d656cc6e0f10eaf1b65ebf WatchSource:0}: Error finding container d5c7bfe522e646c06d7d7f35d8da18325cd495ff10d656cc6e0f10eaf1b65ebf: Status 404 returned error can't find the container with id d5c7bfe522e646c06d7d7f35d8da18325cd495ff10d656cc6e0f10eaf1b65ebf
Jul 19 15:31:53.875231 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:53.875194    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8" event={"ID":"a4ec66e2-085b-4afa-92c8-2397941a882d","Type":"ContainerStarted","Data":"0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad"}
Jul 19 15:31:53.875674 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:53.875241    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8" event={"ID":"a4ec66e2-085b-4afa-92c8-2397941a882d","Type":"ContainerStarted","Data":"d5c7bfe522e646c06d7d7f35d8da18325cd495ff10d656cc6e0f10eaf1b65ebf"}
Jul 19 15:31:53.894176 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:53.894119    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8" podStartSLOduration=1.894103691 podStartE2EDuration="1.894103691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:31:53.89127731 +0000 UTC m=+2715.592725355" watchObservedRunningTime="2026-07-19 15:31:53.894103691 +0000 UTC m=+2715.595551745"
Jul 19 15:31:56.262013 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:31:56.261971    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:32:03.089201 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:03.089160    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:32:03.089744 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:03.089214    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:32:03.102622 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:03.102590    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:32:03.939530 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:03.939491    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:32:19.653740 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:19.653701    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:32:35.060465 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:35.060428    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:32:52.462244 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:32:52.462199    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:33:09.656010 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:33:09.655968    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:33:27.852749 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:33:27.852707    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:33:45.248934 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:33:45.248886    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:33:56.176512 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:33:56.176470    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"]
Jul 19 15:33:56.984726 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:33:56.984683    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"]
Jul 19 15:34:01.265967 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:01.265923    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:34:07.377982 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.377944    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"]
Jul 19 15:34:07.381402 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.381383    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.383893 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.383866    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"sub-test-model-4bc178-kserve-self-signed-certs\""
Jul 19 15:34:07.390249 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.390223    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"]
Jul 19 15:34:07.454224 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.454184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-model-cache\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.454436 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.454233    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-home\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.454436 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.454253    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-dshm\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.454436 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.454322    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01af0d0-473d-4e54-9df5-084077047756-tls-certs\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.454436 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.454387    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p5pg\" (UniqueName: \"kubernetes.io/projected/d01af0d0-473d-4e54-9df5-084077047756-kube-api-access-5p5pg\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.454436 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.454431    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-tmp-dir\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555369 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555316    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-model-cache\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555382    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-home\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-dshm\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555437    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01af0d0-473d-4e54-9df5-084077047756-tls-certs\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555464    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p5pg\" (UniqueName: \"kubernetes.io/projected/d01af0d0-473d-4e54-9df5-084077047756-kube-api-access-5p5pg\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555560 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555501    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-tmp-dir\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555813 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-model-cache\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555873 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555810    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-home\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.555936 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.555920    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-tmp-dir\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.557983 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.557950    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-dshm\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.558227 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.558210    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01af0d0-473d-4e54-9df5-084077047756-tls-certs\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.563400 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.563381    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p5pg\" (UniqueName: \"kubernetes.io/projected/d01af0d0-473d-4e54-9df5-084077047756-kube-api-access-5p5pg\") pod \"sub-test-model-4bc178-kserve-7ff69b69c-ws8tq\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") " pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.692155 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.692050    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:07.841443 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:07.841418    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"]
Jul 19 15:34:07.843232 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:34:07.843204    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd01af0d0_473d_4e54_9df5_084077047756.slice/crio-7717f8dff81cf0b8dea0e78c7e4ac5fa452da2a744367e3802b2b6bc383f157c WatchSource:0}: Error finding container 7717f8dff81cf0b8dea0e78c7e4ac5fa452da2a744367e3802b2b6bc383f157c: Status 404 returned error can't find the container with id 7717f8dff81cf0b8dea0e78c7e4ac5fa452da2a744367e3802b2b6bc383f157c
Jul 19 15:34:08.635724 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:08.635684    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq" event={"ID":"d01af0d0-473d-4e54-9df5-084077047756","Type":"ContainerStarted","Data":"8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383"}
Jul 19 15:34:08.635724 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:08.635731    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq" event={"ID":"d01af0d0-473d-4e54-9df5-084077047756","Type":"ContainerStarted","Data":"7717f8dff81cf0b8dea0e78c7e4ac5fa452da2a744367e3802b2b6bc383f157c"}
Jul 19 15:34:08.652474 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:08.652409    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq" podStartSLOduration=1.652392929 podStartE2EDuration="1.652392929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:34:08.651466948 +0000 UTC m=+2850.352915015" watchObservedRunningTime="2026-07-19 15:34:08.652392929 +0000 UTC m=+2850.353840984"
Jul 19 15:34:11.198260 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.198212    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx" podUID="14e69590-8061-4c6c-a71b-ff443a158a24" containerName="main" containerID="cri-o://4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea" gracePeriod=45
Jul 19 15:34:11.454677 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.454609    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:34:11.487180 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-home\") pod \"14e69590-8061-4c6c-a71b-ff443a158a24\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") "
Jul 19 15:34:11.487386 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487255    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-model-cache\") pod \"14e69590-8061-4c6c-a71b-ff443a158a24\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") "
Jul 19 15:34:11.487386 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487354    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7lwq\" (UniqueName: \"kubernetes.io/projected/14e69590-8061-4c6c-a71b-ff443a158a24-kube-api-access-w7lwq\") pod \"14e69590-8061-4c6c-a71b-ff443a158a24\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") "
Jul 19 15:34:11.487510 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487421    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-tmp-dir\") pod \"14e69590-8061-4c6c-a71b-ff443a158a24\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") "
Jul 19 15:34:11.487510 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487453    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-home" (OuterVolumeSpecName: "home") pod "14e69590-8061-4c6c-a71b-ff443a158a24" (UID: "14e69590-8061-4c6c-a71b-ff443a158a24"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:11.487510 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487468    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-model-cache" (OuterVolumeSpecName: "model-cache") pod "14e69590-8061-4c6c-a71b-ff443a158a24" (UID: "14e69590-8061-4c6c-a71b-ff443a158a24"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:11.487689 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487508    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-dshm\") pod \"14e69590-8061-4c6c-a71b-ff443a158a24\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") "
Jul 19 15:34:11.487689 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487542    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14e69590-8061-4c6c-a71b-ff443a158a24-tls-certs\") pod \"14e69590-8061-4c6c-a71b-ff443a158a24\" (UID: \"14e69590-8061-4c6c-a71b-ff443a158a24\") "
Jul 19 15:34:11.487689 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487676    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "14e69590-8061-4c6c-a71b-ff443a158a24" (UID: "14e69590-8061-4c6c-a71b-ff443a158a24"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:11.487887 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487858    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:11.488015 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487891    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:11.488015 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.487906    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:11.489847 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.489820    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-dshm" (OuterVolumeSpecName: "dshm") pod "14e69590-8061-4c6c-a71b-ff443a158a24" (UID: "14e69590-8061-4c6c-a71b-ff443a158a24"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:11.489939 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.489919    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e69590-8061-4c6c-a71b-ff443a158a24-kube-api-access-w7lwq" (OuterVolumeSpecName: "kube-api-access-w7lwq") pod "14e69590-8061-4c6c-a71b-ff443a158a24" (UID: "14e69590-8061-4c6c-a71b-ff443a158a24"). InnerVolumeSpecName "kube-api-access-w7lwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:34:11.490001 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.489933    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e69590-8061-4c6c-a71b-ff443a158a24-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "14e69590-8061-4c6c-a71b-ff443a158a24" (UID: "14e69590-8061-4c6c-a71b-ff443a158a24"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:34:11.561161 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.560061    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:34:11.588906 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.588868    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14e69590-8061-4c6c-a71b-ff443a158a24-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:11.588906 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.588905    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14e69590-8061-4c6c-a71b-ff443a158a24-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:11.589111 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.588921    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7lwq\" (UniqueName: \"kubernetes.io/projected/14e69590-8061-4c6c-a71b-ff443a158a24-kube-api-access-w7lwq\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:11.654407 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.654374    2572 generic.go:356] "Generic (PLEG): container finished" podID="14e69590-8061-4c6c-a71b-ff443a158a24" containerID="4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea" exitCode=0
Jul 19 15:34:11.654574 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.654441    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"
Jul 19 15:34:11.654574 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.654463    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx" event={"ID":"14e69590-8061-4c6c-a71b-ff443a158a24","Type":"ContainerDied","Data":"4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea"}
Jul 19 15:34:11.654574 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.654514    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx" event={"ID":"14e69590-8061-4c6c-a71b-ff443a158a24","Type":"ContainerDied","Data":"8400e70a04016cb6c2916e9920e4c46a40f2198b05eb84f2995198a9a70d6993"}
Jul 19 15:34:11.654574 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.654537    2572 scope.go:117] "RemoveContainer" containerID="4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea"
Jul 19 15:34:11.664253 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.664235    2572 scope.go:117] "RemoveContainer" containerID="4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea"
Jul 19 15:34:11.664588 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:34:11.664569    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea\": container with ID starting with 4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea not found: ID does not exist" containerID="4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea"
Jul 19 15:34:11.664645 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.664599    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea"} err="failed to get container status \"4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea\": rpc error: code = NotFound desc = could not find container \"4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea\": container with ID starting with 4114439d05928e777719ea9db66edfc62388726a9049703c936de04a4a9f43ea not found: ID does not exist"
Jul 19 15:34:11.675032 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.675008    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"]
Jul 19 15:34:11.677136 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:11.677114    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/auth-test-model-4bc178-kserve-865f674d59-l85hx"]
Jul 19 15:34:12.006614 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.006550    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8" podUID="a4ec66e2-085b-4afa-92c8-2397941a882d" containerName="main" containerID="cri-o://0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad" gracePeriod=45
Jul 19 15:34:12.260542 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.260468    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:34:12.295399 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295357    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-tmp-dir\") pod \"a4ec66e2-085b-4afa-92c8-2397941a882d\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") "
Jul 19 15:34:12.295554 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295418    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-home\") pod \"a4ec66e2-085b-4afa-92c8-2397941a882d\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") "
Jul 19 15:34:12.295554 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295458    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlh7b\" (UniqueName: \"kubernetes.io/projected/a4ec66e2-085b-4afa-92c8-2397941a882d-kube-api-access-jlh7b\") pod \"a4ec66e2-085b-4afa-92c8-2397941a882d\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") "
Jul 19 15:34:12.295554 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295511    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-dshm\") pod \"a4ec66e2-085b-4afa-92c8-2397941a882d\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") "
Jul 19 15:34:12.295686 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295552    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ec66e2-085b-4afa-92c8-2397941a882d-tls-certs\") pod \"a4ec66e2-085b-4afa-92c8-2397941a882d\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") "
Jul 19 15:34:12.295686 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295623    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-model-cache\") pod \"a4ec66e2-085b-4afa-92c8-2397941a882d\" (UID: \"a4ec66e2-085b-4afa-92c8-2397941a882d\") "
Jul 19 15:34:12.295754 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295696    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-home" (OuterVolumeSpecName: "home") pod "a4ec66e2-085b-4afa-92c8-2397941a882d" (UID: "a4ec66e2-085b-4afa-92c8-2397941a882d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:12.295754 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295703    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4ec66e2-085b-4afa-92c8-2397941a882d" (UID: "a4ec66e2-085b-4afa-92c8-2397941a882d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:12.295979 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295952    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4ec66e2-085b-4afa-92c8-2397941a882d" (UID: "a4ec66e2-085b-4afa-92c8-2397941a882d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:12.296110 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.295964    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:12.296110 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.296010    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:12.297998 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.297974    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4ec66e2-085b-4afa-92c8-2397941a882d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4ec66e2-085b-4afa-92c8-2397941a882d" (UID: "a4ec66e2-085b-4afa-92c8-2397941a882d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:34:12.298281 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.298259    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-dshm" (OuterVolumeSpecName: "dshm") pod "a4ec66e2-085b-4afa-92c8-2397941a882d" (UID: "a4ec66e2-085b-4afa-92c8-2397941a882d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:34:12.298398 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.298276    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4ec66e2-085b-4afa-92c8-2397941a882d-kube-api-access-jlh7b" (OuterVolumeSpecName: "kube-api-access-jlh7b") pod "a4ec66e2-085b-4afa-92c8-2397941a882d" (UID: "a4ec66e2-085b-4afa-92c8-2397941a882d"). InnerVolumeSpecName "kube-api-access-jlh7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:34:12.397274 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.397235    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4ec66e2-085b-4afa-92c8-2397941a882d-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:12.397274 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.397269    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:12.397274 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.397279    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlh7b\" (UniqueName: \"kubernetes.io/projected/a4ec66e2-085b-4afa-92c8-2397941a882d-kube-api-access-jlh7b\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:12.397565 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.397288    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4ec66e2-085b-4afa-92c8-2397941a882d-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:34:12.663299 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.663260    2572 generic.go:356] "Generic (PLEG): container finished" podID="a4ec66e2-085b-4afa-92c8-2397941a882d" containerID="0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad" exitCode=0
Jul 19 15:34:12.663529 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.663310    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"
Jul 19 15:34:12.663529 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.663359    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8" event={"ID":"a4ec66e2-085b-4afa-92c8-2397941a882d","Type":"ContainerDied","Data":"0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad"}
Jul 19 15:34:12.663529 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.663411    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8" event={"ID":"a4ec66e2-085b-4afa-92c8-2397941a882d","Type":"ContainerDied","Data":"d5c7bfe522e646c06d7d7f35d8da18325cd495ff10d656cc6e0f10eaf1b65ebf"}
Jul 19 15:34:12.663529 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.663433    2572 scope.go:117] "RemoveContainer" containerID="0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad"
Jul 19 15:34:12.673246 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.673222    2572 scope.go:117] "RemoveContainer" containerID="0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad"
Jul 19 15:34:12.673633 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:34:12.673611    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad\": container with ID starting with 0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad not found: ID does not exist" containerID="0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad"
Jul 19 15:34:12.673693 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.673644    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad"} err="failed to get container status \"0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad\": rpc error: code = NotFound desc = could not find container \"0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad\": container with ID starting with 0e4d3a9739743d6590dd689b4d273f29185a5d443976aa6a2397efeafadb72ad not found: ID does not exist"
Jul 19 15:34:12.684840 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.684659    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"]
Jul 19 15:34:12.687863 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.687837    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/auth-test-model-ccd7ab-kserve-54bf49644c-p5qz8"]
Jul 19 15:34:12.786518 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.786485    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e69590-8061-4c6c-a71b-ff443a158a24" path="/var/lib/kubelet/pods/14e69590-8061-4c6c-a71b-ff443a158a24/volumes"
Jul 19 15:34:12.786903 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:12.786888    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4ec66e2-085b-4afa-92c8-2397941a882d" path="/var/lib/kubelet/pods/a4ec66e2-085b-4afa-92c8-2397941a882d/volumes"
Jul 19 15:34:17.692218 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:17.692183    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:17.692697 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:17.692237    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:17.705001 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:17.704973    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:18.710976 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:18.710941    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:34:36.478014 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.477968    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"]
Jul 19 15:34:36.478676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.478385    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e69590-8061-4c6c-a71b-ff443a158a24" containerName="main"
Jul 19 15:34:36.478676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.478398    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e69590-8061-4c6c-a71b-ff443a158a24" containerName="main"
Jul 19 15:34:36.478676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.478410    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4ec66e2-085b-4afa-92c8-2397941a882d" containerName="main"
Jul 19 15:34:36.478676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.478416    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4ec66e2-085b-4afa-92c8-2397941a882d" containerName="main"
Jul 19 15:34:36.478676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.478486    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4ec66e2-085b-4afa-92c8-2397941a882d" containerName="main"
Jul 19 15:34:36.478676 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.478494    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e69590-8061-4c6c-a71b-ff443a158a24" containerName="main"
Jul 19 15:34:36.481692 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.481668    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.484281 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.484252    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"kube-root-ca.crt\""
Jul 19 15:34:36.484281 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.484289    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"openshift-service-ca.crt\""
Jul 19 15:34:36.485636 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.485613    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"default-dockercfg-sq9bp\""
Jul 19 15:34:36.485939 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.485635    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"sub-test-model-ccd7ab-kserve-self-signed-certs\""
Jul 19 15:34:36.490570 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.490546    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"]
Jul 19 15:34:36.616233 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.616196    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm69j\" (UniqueName: \"kubernetes.io/projected/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-kube-api-access-bm69j\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.616233 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.616237    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tls-certs\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.616497 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.616321    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-model-cache\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.616497 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.616454    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-home\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.616497 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.616475    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tmp-dir\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.616497 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.616495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-dshm\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.716958 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.716917    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-home\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.716958 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.716958    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tmp-dir\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717192 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717092    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-dshm\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717192 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717158    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm69j\" (UniqueName: \"kubernetes.io/projected/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-kube-api-access-bm69j\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717273 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717204    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tls-certs\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717314 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717299    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-model-cache\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717440 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717409    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-home\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717531 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717433    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tmp-dir\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.717767 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.717744    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-model-cache\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.720038 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.719682    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-dshm\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.720620 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.720596    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tls-certs\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.725875 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.725849    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm69j\" (UniqueName: \"kubernetes.io/projected/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-kube-api-access-bm69j\") pod \"sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") " pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.793380 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.793271    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:36.936270 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:36.936242    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"]
Jul 19 15:34:36.938402 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:34:36.938369    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05cdc61e_27ff_4f7e_b963_ad1d01f308aa.slice/crio-2f1a70583f1da31a7c9b892d19e447e58ec0e14ce935a5791294bf1738aa55d6 WatchSource:0}: Error finding container 2f1a70583f1da31a7c9b892d19e447e58ec0e14ce935a5791294bf1738aa55d6: Status 404 returned error can't find the container with id 2f1a70583f1da31a7c9b892d19e447e58ec0e14ce935a5791294bf1738aa55d6
Jul 19 15:34:37.821075 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:37.821034    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2" event={"ID":"05cdc61e-27ff-4f7e-b963-ad1d01f308aa","Type":"ContainerStarted","Data":"da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1"}
Jul 19 15:34:37.821531 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:37.821084    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2" event={"ID":"05cdc61e-27ff-4f7e-b963-ad1d01f308aa","Type":"ContainerStarted","Data":"2f1a70583f1da31a7c9b892d19e447e58ec0e14ce935a5791294bf1738aa55d6"}
Jul 19 15:34:37.838246 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:37.838192    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2" podStartSLOduration=1.8381743209999999 podStartE2EDuration="1.838174321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:34:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:34:37.83630883 +0000 UTC m=+2879.537756885" watchObservedRunningTime="2026-07-19 15:34:37.838174321 +0000 UTC m=+2879.539622395"
Jul 19 15:34:40.057790 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:40.057755    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:34:46.793749 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:46.793710    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:46.794178 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:46.793767    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:46.806215 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:46.806186    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:46.880667 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:46.880636    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:34:56.654544 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:34:56.654507    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:35:12.656797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:12.656756    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:35:19.453154 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:19.453112    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"]
Jul 19 15:35:20.252624 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:20.252586    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"]
Jul 19 15:35:30.157216 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:30.157174    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:35:34.476263 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.476198    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq" podUID="d01af0d0-473d-4e54-9df5-084077047756" containerName="main" containerID="cri-o://8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383" gracePeriod=45
Jul 19 15:35:34.730189 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.730127    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:35:34.845520 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845490    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-tmp-dir\") pod \"d01af0d0-473d-4e54-9df5-084077047756\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") "
Jul 19 15:35:34.845520 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845529    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-model-cache\") pod \"d01af0d0-473d-4e54-9df5-084077047756\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") "
Jul 19 15:35:34.845797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845554    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01af0d0-473d-4e54-9df5-084077047756-tls-certs\") pod \"d01af0d0-473d-4e54-9df5-084077047756\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") "
Jul 19 15:35:34.845797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845578    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-home\") pod \"d01af0d0-473d-4e54-9df5-084077047756\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") "
Jul 19 15:35:34.845797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-dshm\") pod \"d01af0d0-473d-4e54-9df5-084077047756\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") "
Jul 19 15:35:34.845797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845631    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5p5pg\" (UniqueName: \"kubernetes.io/projected/d01af0d0-473d-4e54-9df5-084077047756-kube-api-access-5p5pg\") pod \"d01af0d0-473d-4e54-9df5-084077047756\" (UID: \"d01af0d0-473d-4e54-9df5-084077047756\") "
Jul 19 15:35:34.845797 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845783    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d01af0d0-473d-4e54-9df5-084077047756" (UID: "d01af0d0-473d-4e54-9df5-084077047756"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:34.846009 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845825    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-model-cache" (OuterVolumeSpecName: "model-cache") pod "d01af0d0-473d-4e54-9df5-084077047756" (UID: "d01af0d0-473d-4e54-9df5-084077047756"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:34.846009 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.845842    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-home" (OuterVolumeSpecName: "home") pod "d01af0d0-473d-4e54-9df5-084077047756" (UID: "d01af0d0-473d-4e54-9df5-084077047756"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:34.846009 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.846000    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:34.846165 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.846020    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:34.846165 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.846031    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:34.848123 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.848083    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01af0d0-473d-4e54-9df5-084077047756-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d01af0d0-473d-4e54-9df5-084077047756" (UID: "d01af0d0-473d-4e54-9df5-084077047756"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:35:34.848123 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.848107    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01af0d0-473d-4e54-9df5-084077047756-kube-api-access-5p5pg" (OuterVolumeSpecName: "kube-api-access-5p5pg") pod "d01af0d0-473d-4e54-9df5-084077047756" (UID: "d01af0d0-473d-4e54-9df5-084077047756"). InnerVolumeSpecName "kube-api-access-5p5pg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:35:34.848123 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.848092    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-dshm" (OuterVolumeSpecName: "dshm") pod "d01af0d0-473d-4e54-9df5-084077047756" (UID: "d01af0d0-473d-4e54-9df5-084077047756"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:34.947085 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.947043    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01af0d0-473d-4e54-9df5-084077047756-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:34.947085 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.947078    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01af0d0-473d-4e54-9df5-084077047756-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:34.947085 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:34.947088    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p5pg\" (UniqueName: \"kubernetes.io/projected/d01af0d0-473d-4e54-9df5-084077047756-kube-api-access-5p5pg\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:35.137991 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.137958    2572 generic.go:356] "Generic (PLEG): container finished" podID="d01af0d0-473d-4e54-9df5-084077047756" containerID="8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383" exitCode=0
Jul 19 15:35:35.138157 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.138004    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"
Jul 19 15:35:35.138157 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.138053    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq" event={"ID":"d01af0d0-473d-4e54-9df5-084077047756","Type":"ContainerDied","Data":"8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383"}
Jul 19 15:35:35.138157 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.138105    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq" event={"ID":"d01af0d0-473d-4e54-9df5-084077047756","Type":"ContainerDied","Data":"7717f8dff81cf0b8dea0e78c7e4ac5fa452da2a744367e3802b2b6bc383f157c"}
Jul 19 15:35:35.138157 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.138125    2572 scope.go:117] "RemoveContainer" containerID="8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383"
Jul 19 15:35:35.148178 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.148157    2572 scope.go:117] "RemoveContainer" containerID="8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383"
Jul 19 15:35:35.148502 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:35:35.148472    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383\": container with ID starting with 8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383 not found: ID does not exist" containerID="8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383"
Jul 19 15:35:35.148583 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.148514    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383"} err="failed to get container status \"8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383\": rpc error: code = NotFound desc = could not find container \"8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383\": container with ID starting with 8f53d7d7bfb7eff6515c0cd6d497b87b98fe40599b2a55ccb8aebc9e0aa2e383 not found: ID does not exist"
Jul 19 15:35:35.159102 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.159075    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"]
Jul 19 15:35:35.162991 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.162959    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/sub-test-model-4bc178-kserve-7ff69b69c-ws8tq"]
Jul 19 15:35:35.273820 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.273780    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2" podUID="05cdc61e-27ff-4f7e-b963-ad1d01f308aa" containerName="main" containerID="cri-o://da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1" gracePeriod=45
Jul 19 15:35:35.529918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.529895    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:35:35.654133 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654094    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-model-cache\") pod \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") "
Jul 19 15:35:35.654133 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654134    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-dshm\") pod \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") "
Jul 19 15:35:35.654448 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654164    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-home\") pod \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") "
Jul 19 15:35:35.654448 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654185    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tmp-dir\") pod \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") "
Jul 19 15:35:35.654448 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654217    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bm69j\" (UniqueName: \"kubernetes.io/projected/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-kube-api-access-bm69j\") pod \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") "
Jul 19 15:35:35.654448 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654277    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tls-certs\") pod \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\" (UID: \"05cdc61e-27ff-4f7e-b963-ad1d01f308aa\") "
Jul 19 15:35:35.654651 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654461    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-model-cache" (OuterVolumeSpecName: "model-cache") pod "05cdc61e-27ff-4f7e-b963-ad1d01f308aa" (UID: "05cdc61e-27ff-4f7e-b963-ad1d01f308aa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:35.654651 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654479    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-home" (OuterVolumeSpecName: "home") pod "05cdc61e-27ff-4f7e-b963-ad1d01f308aa" (UID: "05cdc61e-27ff-4f7e-b963-ad1d01f308aa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:35.654651 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654544    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05cdc61e-27ff-4f7e-b963-ad1d01f308aa" (UID: "05cdc61e-27ff-4f7e-b963-ad1d01f308aa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:35.654768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654687    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:35.654768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654705    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:35.654768 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.654717    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:35.656564 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.656541    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-dshm" (OuterVolumeSpecName: "dshm") pod "05cdc61e-27ff-4f7e-b963-ad1d01f308aa" (UID: "05cdc61e-27ff-4f7e-b963-ad1d01f308aa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:35:35.656564 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.656555    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-kube-api-access-bm69j" (OuterVolumeSpecName: "kube-api-access-bm69j") pod "05cdc61e-27ff-4f7e-b963-ad1d01f308aa" (UID: "05cdc61e-27ff-4f7e-b963-ad1d01f308aa"). InnerVolumeSpecName "kube-api-access-bm69j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:35:35.656727 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.656634    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05cdc61e-27ff-4f7e-b963-ad1d01f308aa" (UID: "05cdc61e-27ff-4f7e-b963-ad1d01f308aa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:35:35.756081 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.755981    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm69j\" (UniqueName: \"kubernetes.io/projected/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-kube-api-access-bm69j\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:35.756081 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.756015    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:35.756081 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:35.756026    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05cdc61e-27ff-4f7e-b963-ad1d01f308aa-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:35:36.146851 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.146814    2572 generic.go:356] "Generic (PLEG): container finished" podID="05cdc61e-27ff-4f7e-b963-ad1d01f308aa" containerID="da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1" exitCode=0
Jul 19 15:35:36.147052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.146863    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"
Jul 19 15:35:36.147052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.146886    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2" event={"ID":"05cdc61e-27ff-4f7e-b963-ad1d01f308aa","Type":"ContainerDied","Data":"da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1"}
Jul 19 15:35:36.147052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.146920    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2" event={"ID":"05cdc61e-27ff-4f7e-b963-ad1d01f308aa","Type":"ContainerDied","Data":"2f1a70583f1da31a7c9b892d19e447e58ec0e14ce935a5791294bf1738aa55d6"}
Jul 19 15:35:36.147052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.146936    2572 scope.go:117] "RemoveContainer" containerID="da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1"
Jul 19 15:35:36.157084 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.157064    2572 scope.go:117] "RemoveContainer" containerID="da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1"
Jul 19 15:35:36.157412 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:35:36.157385    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1\": container with ID starting with da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1 not found: ID does not exist" containerID="da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1"
Jul 19 15:35:36.157527 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.157418    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1"} err="failed to get container status \"da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1\": rpc error: code = NotFound desc = could not find container \"da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1\": container with ID starting with da6035a69c052db467d126188ef2a1523432523728d6debf77b05409d379e0f1 not found: ID does not exist"
Jul 19 15:35:36.167700 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.167670    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"]
Jul 19 15:35:36.171903 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.171875    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/sub-test-model-ccd7ab-kserve-7d94fcf847-2bbk2"]
Jul 19 15:35:36.792246 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.792209    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05cdc61e-27ff-4f7e-b963-ad1d01f308aa" path="/var/lib/kubelet/pods/05cdc61e-27ff-4f7e-b963-ad1d01f308aa/volumes"
Jul 19 15:35:36.792653 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:36.792628    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d01af0d0-473d-4e54-9df5-084077047756" path="/var/lib/kubelet/pods/d01af0d0-473d-4e54-9df5-084077047756/volumes"
Jul 19 15:35:39.102957 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:39.102921    2572 scope.go:117] "RemoveContainer" containerID="66d57724d1a91d0d9cad4d4afe657854cebdf0df23ce318108fd5ebbb56f4ae9"
Jul 19 15:35:53.678487 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.678447    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"]
Jul 19 15:35:53.679121 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.678997    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01af0d0-473d-4e54-9df5-084077047756" containerName="main"
Jul 19 15:35:53.679121 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.679022    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01af0d0-473d-4e54-9df5-084077047756" containerName="main"
Jul 19 15:35:53.679121 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.679042    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05cdc61e-27ff-4f7e-b963-ad1d01f308aa" containerName="main"
Jul 19 15:35:53.679121 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.679052    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="05cdc61e-27ff-4f7e-b963-ad1d01f308aa" containerName="main"
Jul 19 15:35:53.679414 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.679163    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="05cdc61e-27ff-4f7e-b963-ad1d01f308aa" containerName="main"
Jul 19 15:35:53.679414 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.679176    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01af0d0-473d-4e54-9df5-084077047756" containerName="main"
Jul 19 15:35:53.682412 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.682388    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.685048 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.685020    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"openshift-service-ca.crt\""
Jul 19 15:35:53.685191 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.685135    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"kube-root-ca.crt\""
Jul 19 15:35:53.686260 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.686242    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"default-dockercfg-clv5l\""
Jul 19 15:35:53.686400 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.686287    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4bc178\"/\"rate-test-model-4bc178-kserve-self-signed-certs\""
Jul 19 15:35:53.691841 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.691813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"]
Jul 19 15:35:53.719829 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.719792    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs4x6\" (UniqueName: \"kubernetes.io/projected/7bdca071-ef93-4317-a687-0fdb2e0665ba-kube-api-access-fs4x6\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.720033 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.719872    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-home\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.720033 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.719913    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-dshm\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.720033 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.719945    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdca071-ef93-4317-a687-0fdb2e0665ba-tls-certs\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.720033 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.719977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-model-cache\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.720033 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.720001    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-tmp-dir\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821237 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821189    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs4x6\" (UniqueName: \"kubernetes.io/projected/7bdca071-ef93-4317-a687-0fdb2e0665ba-kube-api-access-fs4x6\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821256    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-home\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-dshm\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdca071-ef93-4317-a687-0fdb2e0665ba-tls-certs\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821704 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821537    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-model-cache\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821704 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-tmp-dir\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821827 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821793    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-home\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821887 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821871    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-tmp-dir\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.821975 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.821952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-model-cache\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.823942 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.823922    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-dshm\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.824119 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.824103    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdca071-ef93-4317-a687-0fdb2e0665ba-tls-certs\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.828779 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.828754    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs4x6\" (UniqueName: \"kubernetes.io/projected/7bdca071-ef93-4317-a687-0fdb2e0665ba-kube-api-access-fs4x6\") pod \"rate-test-model-4bc178-kserve-659684dcff-cwld4\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") " pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:53.993370 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:53.993246    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:35:54.137282 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:54.137250    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"]
Jul 19 15:35:54.139958 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:35:54.139928    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bdca071_ef93_4317_a687_0fdb2e0665ba.slice/crio-08a7f8a817561e75aff4aad98a7f8a9c6d366e27b72505445cfa69bd2cb30aca WatchSource:0}: Error finding container 08a7f8a817561e75aff4aad98a7f8a9c6d366e27b72505445cfa69bd2cb30aca: Status 404 returned error can't find the container with id 08a7f8a817561e75aff4aad98a7f8a9c6d366e27b72505445cfa69bd2cb30aca
Jul 19 15:35:54.250857 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:54.250766    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4" event={"ID":"7bdca071-ef93-4317-a687-0fdb2e0665ba","Type":"ContainerStarted","Data":"a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead"}
Jul 19 15:35:54.250857 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:54.250810    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4" event={"ID":"7bdca071-ef93-4317-a687-0fdb2e0665ba","Type":"ContainerStarted","Data":"08a7f8a817561e75aff4aad98a7f8a9c6d366e27b72505445cfa69bd2cb30aca"}
Jul 19 15:35:54.268791 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:54.268733    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4" podStartSLOduration=1.268710427 podStartE2EDuration="1.268710427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:35:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:35:54.266015299 +0000 UTC m=+2955.967463355" watchObservedRunningTime="2026-07-19 15:35:54.268710427 +0000 UTC m=+2955.970158480"
Jul 19 15:35:57.355617 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:35:57.355578    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:36:03.993764 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:03.993658    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:36:03.993764 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:03.993715    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:36:04.006692 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:04.006663    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:36:04.322208 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:04.322117    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:36:10.675347 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.675291    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"]
Jul 19 15:36:10.679328 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.679301    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.681977 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.681953    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"kube-root-ca.crt\""
Jul 19 15:36:10.683213 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.683194    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"openshift-service-ca.crt\""
Jul 19 15:36:10.683324 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.683287    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"default-dockercfg-sq9bp\""
Jul 19 15:36:10.683406 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.683287    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ccd7ab\"/\"rate-test-model-ccd7ab-kserve-self-signed-certs\""
Jul 19 15:36:10.689365 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.689312    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"]
Jul 19 15:36:10.770249 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.770202    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-dshm\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.770504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.770283    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-home\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.770504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.770319    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ntn4\" (UniqueName: \"kubernetes.io/projected/18e6b54f-9a14-4505-8869-1c5e01069720-kube-api-access-4ntn4\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.770504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.770396    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e6b54f-9a14-4505-8869-1c5e01069720-tls-certs\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.770665 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.770493    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-model-cache\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.770665 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.770528    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-tmp-dir\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.871939 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.871904    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-model-cache\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.871939 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.871946    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-tmp-dir\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872219 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872011    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-dshm\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872219 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872049    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-home\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872219 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ntn4\" (UniqueName: \"kubernetes.io/projected/18e6b54f-9a14-4505-8869-1c5e01069720-kube-api-access-4ntn4\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872219 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872091    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e6b54f-9a14-4505-8869-1c5e01069720-tls-certs\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872525 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872459    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-model-cache\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872525 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872475    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-tmp-dir\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.872597 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.872521    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-home\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.874659 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.874638    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-dshm\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.874822 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.874804    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e6b54f-9a14-4505-8869-1c5e01069720-tls-certs\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.880758 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.880734    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ntn4\" (UniqueName: \"kubernetes.io/projected/18e6b54f-9a14-4505-8869-1c5e01069720-kube-api-access-4ntn4\") pod \"rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") " pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:10.991413 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:10.991303    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:11.131865 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:11.131830    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"]
Jul 19 15:36:11.133574 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:36:11.133549    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e6b54f_9a14_4505_8869_1c5e01069720.slice/crio-4f71672f2179e68dcd152f9b7853483040076a24c1f4b414c93d3411bc31fa07 WatchSource:0}: Error finding container 4f71672f2179e68dcd152f9b7853483040076a24c1f4b414c93d3411bc31fa07: Status 404 returned error can't find the container with id 4f71672f2179e68dcd152f9b7853483040076a24c1f4b414c93d3411bc31fa07
Jul 19 15:36:11.354531 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:11.354492    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk" event={"ID":"18e6b54f-9a14-4505-8869-1c5e01069720","Type":"ContainerStarted","Data":"1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be"}
Jul 19 15:36:11.354531 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:11.354537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk" event={"ID":"18e6b54f-9a14-4505-8869-1c5e01069720","Type":"ContainerStarted","Data":"4f71672f2179e68dcd152f9b7853483040076a24c1f4b414c93d3411bc31fa07"}
Jul 19 15:36:11.378221 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:11.378150    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk" podStartSLOduration=1.37812434 podStartE2EDuration="1.37812434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:36:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:36:11.369995398 +0000 UTC m=+2973.071443455" watchObservedRunningTime="2026-07-19 15:36:11.37812434 +0000 UTC m=+2973.079572396"
Jul 19 15:36:14.551960 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:14.551909    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:36:20.991635 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:20.991573    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:20.991635 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:20.991655    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:21.004908 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:21.004876    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:21.421034 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:21.421004    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:36:28.655180 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:28.655144    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:36:39.039786 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:39.039757    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:36:39.053310 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:39.053284    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wfm6z_25c3a9a1-c4a8-4705-a9bc-12b106f29cfd/ovn-acl-logging/0.log"
Jul 19 15:36:39.133558 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:39.133526    2572 scope.go:117] "RemoveContainer" containerID="502cbf0ba595a1bab411eff7434235e59097f104593b7a8463e581760a5f6ecc"
Jul 19 15:36:39.142223 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:39.142201    2572 scope.go:117] "RemoveContainer" containerID="abd401794eb8f12072b09321e6a9c121ccc0f22e1529a71b047e2b30cfb06038"
Jul 19 15:36:39.150416 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:39.150396    2572 scope.go:117] "RemoveContainer" containerID="8451446f816af3158ddce61a44293942ab80df91eaed45be5187653f27b6cd0a"
Jul 19 15:36:39.158314 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:39.158293    2572 scope.go:117] "RemoveContainer" containerID="68748bc96827fb296cf0d1c1d38e1d3c8b0139ecaa7feaaea020dd0de6977519"
Jul 19 15:36:43.757813 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:43.757770    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:36:50.051514 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:50.051479    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"]
Jul 19 15:36:50.870509 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:50.870470    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"]
Jul 19 15:36:51.859178 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:51.859141    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7555-140401905893552"]
Jul 19 15:36:51.862506 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:51.862489    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:51.865157 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:51.865137    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rrp6j\""
Jul 19 15:36:51.868937 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:51.868914    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401905893552"]
Jul 19 15:36:51.944688 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:51.944641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptzs6\" (UniqueName: \"kubernetes.io/projected/0b1d19d8-7300-40ac-8ef9-2861038fcfff-kube-api-access-ptzs6\") pod \"test-curl-7555-140401905893552\" (UID: \"0b1d19d8-7300-40ac-8ef9-2861038fcfff\") " pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:52.046091 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:52.046051    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptzs6\" (UniqueName: \"kubernetes.io/projected/0b1d19d8-7300-40ac-8ef9-2861038fcfff-kube-api-access-ptzs6\") pod \"test-curl-7555-140401905893552\" (UID: \"0b1d19d8-7300-40ac-8ef9-2861038fcfff\") " pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:52.053955 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:52.053921    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptzs6\" (UniqueName: \"kubernetes.io/projected/0b1d19d8-7300-40ac-8ef9-2861038fcfff-kube-api-access-ptzs6\") pod \"test-curl-7555-140401905893552\" (UID: \"0b1d19d8-7300-40ac-8ef9-2861038fcfff\") " pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:52.173520 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:52.173407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:52.311297 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:52.311267    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401905893552"]
Jul 19 15:36:52.313399 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:36:52.313366    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b1d19d8_7300_40ac_8ef9_2861038fcfff.slice/crio-ca0195c94ff829aac34df4fac4bf7b94a642be38b5ec289aba900a15fe9db576 WatchSource:0}: Error finding container ca0195c94ff829aac34df4fac4bf7b94a642be38b5ec289aba900a15fe9db576: Status 404 returned error can't find the container with id ca0195c94ff829aac34df4fac4bf7b94a642be38b5ec289aba900a15fe9db576
Jul 19 15:36:52.314725 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:52.314707    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 19 15:36:52.589117 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:52.589086    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401905893552" event={"ID":"0b1d19d8-7300-40ac-8ef9-2861038fcfff","Type":"ContainerStarted","Data":"ca0195c94ff829aac34df4fac4bf7b94a642be38b5ec289aba900a15fe9db576"}
Jul 19 15:36:53.595331 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:53.595292    2572 generic.go:356] "Generic (PLEG): container finished" podID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" containerID="0401249b24462795a6c48044e8099541a5241b2914c3c368697b9023208fe379" exitCode=0
Jul 19 15:36:53.595767 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:53.595375    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401905893552" event={"ID":"0b1d19d8-7300-40ac-8ef9-2861038fcfff","Type":"ContainerDied","Data":"0401249b24462795a6c48044e8099541a5241b2914c3c368697b9023208fe379"}
Jul 19 15:36:54.739832 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.739808    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:54.771827 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.771789    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptzs6\" (UniqueName: \"kubernetes.io/projected/0b1d19d8-7300-40ac-8ef9-2861038fcfff-kube-api-access-ptzs6\") pod \"0b1d19d8-7300-40ac-8ef9-2861038fcfff\" (UID: \"0b1d19d8-7300-40ac-8ef9-2861038fcfff\") "
Jul 19 15:36:54.774402 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.774369    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b1d19d8-7300-40ac-8ef9-2861038fcfff-kube-api-access-ptzs6" (OuterVolumeSpecName: "kube-api-access-ptzs6") pod "0b1d19d8-7300-40ac-8ef9-2861038fcfff" (UID: "0b1d19d8-7300-40ac-8ef9-2861038fcfff"). InnerVolumeSpecName "kube-api-access-ptzs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:36:54.774548 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.774419    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7555-140401905893552_0b1d19d8-7300-40ac-8ef9-2861038fcfff/test-curl-7555-140401905893552/0.log"
Jul 19 15:36:54.798880 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.798857    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7555-140401905893552"]
Jul 19 15:36:54.802055 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.802030    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7555-140401905893552"]
Jul 19 15:36:54.873105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.873010    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptzs6\" (UniqueName: \"kubernetes.io/projected/0b1d19d8-7300-40ac-8ef9-2861038fcfff-kube-api-access-ptzs6\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:36:54.925821 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.925787    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7555-140401905896880"]
Jul 19 15:36:54.926189 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.926175    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" containerName="test-curl-7555-140401905893552"
Jul 19 15:36:54.926247 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.926191    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" containerName="test-curl-7555-140401905893552"
Jul 19 15:36:54.926303 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.926293    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" containerName="test-curl-7555-140401905893552"
Jul 19 15:36:54.929375 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.929327    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:54.935786 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.935752    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401905896880"]
Jul 19 15:36:54.973741 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:54.973698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwwm9\" (UniqueName: \"kubernetes.io/projected/bcf162d4-450f-4b42-b596-09481440b20f-kube-api-access-qwwm9\") pod \"test-curl-7555-140401905896880\" (UID: \"bcf162d4-450f-4b42-b596-09481440b20f\") " pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:55.074748 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.074703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwwm9\" (UniqueName: \"kubernetes.io/projected/bcf162d4-450f-4b42-b596-09481440b20f-kube-api-access-qwwm9\") pod \"test-curl-7555-140401905896880\" (UID: \"bcf162d4-450f-4b42-b596-09481440b20f\") " pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:55.083613 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.083588    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwwm9\" (UniqueName: \"kubernetes.io/projected/bcf162d4-450f-4b42-b596-09481440b20f-kube-api-access-qwwm9\") pod \"test-curl-7555-140401905896880\" (UID: \"bcf162d4-450f-4b42-b596-09481440b20f\") " pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:55.239869 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.239770    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:55.377409 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.377375    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401905896880"]
Jul 19 15:36:55.379154 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:36:55.379132    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcf162d4_450f_4b42_b596_09481440b20f.slice/crio-40a14b6fb042e9326fabd98975d1d5a457255d0fd5d32010ece5e4e3a3fb8fd3 WatchSource:0}: Error finding container 40a14b6fb042e9326fabd98975d1d5a457255d0fd5d32010ece5e4e3a3fb8fd3: Status 404 returned error can't find the container with id 40a14b6fb042e9326fabd98975d1d5a457255d0fd5d32010ece5e4e3a3fb8fd3
Jul 19 15:36:55.608158 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.608120    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401905896880" event={"ID":"bcf162d4-450f-4b42-b596-09481440b20f","Type":"ContainerStarted","Data":"40a14b6fb042e9326fabd98975d1d5a457255d0fd5d32010ece5e4e3a3fb8fd3"}
Jul 19 15:36:55.609507 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.609483    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:55.609507 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:55.609498    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca0195c94ff829aac34df4fac4bf7b94a642be38b5ec289aba900a15fe9db576"
Jul 19 15:36:55.614848 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:36:55.614820    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7555-140401905893552\" is forbidden: User \"system:node:ip-10-0-135-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-98.ec2.internal' and this object" podUID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:56.616727 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:56.616690    2572 generic.go:356] "Generic (PLEG): container finished" podID="bcf162d4-450f-4b42-b596-09481440b20f" containerID="3e200e3f578926f6fbe6e90eb7452fa26fe81f82b9d7abc0b06053df273189bb" exitCode=0
Jul 19 15:36:56.617131 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:56.616753    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401905896880" event={"ID":"bcf162d4-450f-4b42-b596-09481440b20f","Type":"ContainerDied","Data":"3e200e3f578926f6fbe6e90eb7452fa26fe81f82b9d7abc0b06053df273189bb"}
Jul 19 15:36:56.631722 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:36:56.631680    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7555-140401905893552\" is forbidden: User \"system:node:ip-10-0-135-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-98.ec2.internal' and this object" podUID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" pod="opendatahub/test-curl-7555-140401905893552"
Jul 19 15:36:56.787171 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:56.787132    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b1d19d8-7300-40ac-8ef9-2861038fcfff" path="/var/lib/kubelet/pods/0b1d19d8-7300-40ac-8ef9-2861038fcfff/volumes"
Jul 19 15:36:57.761399 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.761374    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:57.779868 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.779844    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7555-140401905896880_bcf162d4-450f-4b42-b596-09481440b20f/test-curl-7555-140401905896880/0.log"
Jul 19 15:36:57.798301 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.798266    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwwm9\" (UniqueName: \"kubernetes.io/projected/bcf162d4-450f-4b42-b596-09481440b20f-kube-api-access-qwwm9\") pod \"bcf162d4-450f-4b42-b596-09481440b20f\" (UID: \"bcf162d4-450f-4b42-b596-09481440b20f\") "
Jul 19 15:36:57.800830 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.800801    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcf162d4-450f-4b42-b596-09481440b20f-kube-api-access-qwwm9" (OuterVolumeSpecName: "kube-api-access-qwwm9") pod "bcf162d4-450f-4b42-b596-09481440b20f" (UID: "bcf162d4-450f-4b42-b596-09481440b20f"). InnerVolumeSpecName "kube-api-access-qwwm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:36:57.805371 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.805317    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7555-140401905896880"]
Jul 19 15:36:57.808973 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.808952    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7555-140401905896880"]
Jul 19 15:36:57.899684 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.899591    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwwm9\" (UniqueName: \"kubernetes.io/projected/bcf162d4-450f-4b42-b596-09481440b20f-kube-api-access-qwwm9\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:36:57.964928 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.964893    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:36:57.965278 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.965265    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcf162d4-450f-4b42-b596-09481440b20f" containerName="test-curl-7555-140401905896880"
Jul 19 15:36:57.965318 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.965280    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcf162d4-450f-4b42-b596-09481440b20f" containerName="test-curl-7555-140401905896880"
Jul 19 15:36:57.965375 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.965358    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcf162d4-450f-4b42-b596-09481440b20f" containerName="test-curl-7555-140401905896880"
Jul 19 15:36:57.968242 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.968224    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:36:57.974472 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:57.974445    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:36:58.000485 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.000435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4zlf\" (UniqueName: \"kubernetes.io/projected/8cf317dc-8580-46c2-83ca-a079fdf8609e-kube-api-access-c4zlf\") pod \"test-curl-7555-140401906910384\" (UID: \"8cf317dc-8580-46c2-83ca-a079fdf8609e\") " pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:36:58.102034 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.101983    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4zlf\" (UniqueName: \"kubernetes.io/projected/8cf317dc-8580-46c2-83ca-a079fdf8609e-kube-api-access-c4zlf\") pod \"test-curl-7555-140401906910384\" (UID: \"8cf317dc-8580-46c2-83ca-a079fdf8609e\") " pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:36:58.111169 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.111142    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4zlf\" (UniqueName: \"kubernetes.io/projected/8cf317dc-8580-46c2-83ca-a079fdf8609e-kube-api-access-c4zlf\") pod \"test-curl-7555-140401906910384\" (UID: \"8cf317dc-8580-46c2-83ca-a079fdf8609e\") " pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:36:58.279007 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.278914    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:36:58.425602 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.425570    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:36:58.427403 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:36:58.427379    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cf317dc_8580_46c2_83ca_a079fdf8609e.slice/crio-2086b6b0291b75a84cdcf2f3c92888e1e38253ecf288628c0be52dfa14f57a42 WatchSource:0}: Error finding container 2086b6b0291b75a84cdcf2f3c92888e1e38253ecf288628c0be52dfa14f57a42: Status 404 returned error can't find the container with id 2086b6b0291b75a84cdcf2f3c92888e1e38253ecf288628c0be52dfa14f57a42
Jul 19 15:36:58.630615 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.630555    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906910384" event={"ID":"8cf317dc-8580-46c2-83ca-a079fdf8609e","Type":"ContainerStarted","Data":"2086b6b0291b75a84cdcf2f3c92888e1e38253ecf288628c0be52dfa14f57a42"}
Jul 19 15:36:58.632021 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.631999    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:58.632149 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.632031    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40a14b6fb042e9326fabd98975d1d5a457255d0fd5d32010ece5e4e3a3fb8fd3"
Jul 19 15:36:58.646040 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:36:58.646008    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7555-140401905896880\" is forbidden: User \"system:node:ip-10-0-135-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-98.ec2.internal' and this object" podUID="bcf162d4-450f-4b42-b596-09481440b20f" pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:58.786177 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:36:58.786135    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7555-140401905896880\" is forbidden: User \"system:node:ip-10-0-135-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-98.ec2.internal' and this object" podUID="bcf162d4-450f-4b42-b596-09481440b20f" pod="opendatahub/test-curl-7555-140401905896880"
Jul 19 15:36:58.788192 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:58.788167    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf162d4-450f-4b42-b596-09481440b20f" path="/var/lib/kubelet/pods/bcf162d4-450f-4b42-b596-09481440b20f/volumes"
Jul 19 15:36:59.638275 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:59.638235    2572 generic.go:356] "Generic (PLEG): container finished" podID="8cf317dc-8580-46c2-83ca-a079fdf8609e" containerID="735962b3ddb78ad36cd01043af16d48704636f13626ce29d3ec099b5330679d1" exitCode=0
Jul 19 15:36:59.638493 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:36:59.638325    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906910384" event={"ID":"8cf317dc-8580-46c2-83ca-a079fdf8609e","Type":"ContainerDied","Data":"735962b3ddb78ad36cd01043af16d48704636f13626ce29d3ec099b5330679d1"}
Jul 19 15:37:00.786525 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.786497    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:00.807143 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.807120    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7555-140401906910384_8cf317dc-8580-46c2-83ca-a079fdf8609e/test-curl-7555-140401906910384/0.log"
Jul 19 15:37:00.827662 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.827618    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4zlf\" (UniqueName: \"kubernetes.io/projected/8cf317dc-8580-46c2-83ca-a079fdf8609e-kube-api-access-c4zlf\") pod \"8cf317dc-8580-46c2-83ca-a079fdf8609e\" (UID: \"8cf317dc-8580-46c2-83ca-a079fdf8609e\") "
Jul 19 15:37:00.830423 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.830391    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf317dc-8580-46c2-83ca-a079fdf8609e-kube-api-access-c4zlf" (OuterVolumeSpecName: "kube-api-access-c4zlf") pod "8cf317dc-8580-46c2-83ca-a079fdf8609e" (UID: "8cf317dc-8580-46c2-83ca-a079fdf8609e"). InnerVolumeSpecName "kube-api-access-c4zlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:00.831212 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.831191    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:00.833304 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.833283    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:00.928874 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.928781    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4zlf\" (UniqueName: \"kubernetes.io/projected/8cf317dc-8580-46c2-83ca-a079fdf8609e-kube-api-access-c4zlf\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:00.952072 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.952036    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:00.952425 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.952411    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf317dc-8580-46c2-83ca-a079fdf8609e" containerName="test-curl-7555-140401906910384"
Jul 19 15:37:00.952469 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.952427    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf317dc-8580-46c2-83ca-a079fdf8609e" containerName="test-curl-7555-140401906910384"
Jul 19 15:37:00.952520 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.952496    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf317dc-8580-46c2-83ca-a079fdf8609e" containerName="test-curl-7555-140401906910384"
Jul 19 15:37:00.955608 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.955584    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:00.962610 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:00.962586    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:01.029892 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.029850    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbgbn\" (UniqueName: \"kubernetes.io/projected/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2-kube-api-access-zbgbn\") pod \"test-curl-7555-140401906910384\" (UID: \"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2\") " pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:01.130801 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.130763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbgbn\" (UniqueName: \"kubernetes.io/projected/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2-kube-api-access-zbgbn\") pod \"test-curl-7555-140401906910384\" (UID: \"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2\") " pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:01.141421 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.141390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbgbn\" (UniqueName: \"kubernetes.io/projected/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2-kube-api-access-zbgbn\") pod \"test-curl-7555-140401906910384\" (UID: \"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2\") " pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:01.154978 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.154945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:37:01.267242 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.267153    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:01.409434 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.409409    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:01.410352 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:37:01.410305    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8db7f5ef_0b89_414b_8ba5_a5b70a88e4b2.slice/crio-8abbf981b9aedbe1f6fc6a40382e9081c6d65cbd1e37a3ea6beeb0631869e4a8 WatchSource:0}: Error finding container 8abbf981b9aedbe1f6fc6a40382e9081c6d65cbd1e37a3ea6beeb0631869e4a8: Status 404 returned error can't find the container with id 8abbf981b9aedbe1f6fc6a40382e9081c6d65cbd1e37a3ea6beeb0631869e4a8
Jul 19 15:37:01.650456 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.650410    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906910384" event={"ID":"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2","Type":"ContainerStarted","Data":"8abbf981b9aedbe1f6fc6a40382e9081c6d65cbd1e37a3ea6beeb0631869e4a8"}
Jul 19 15:37:01.651640 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.651615    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2086b6b0291b75a84cdcf2f3c92888e1e38253ecf288628c0be52dfa14f57a42"
Jul 19 15:37:01.651640 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.651634    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:01.663787 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:01.663753    2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7555-140401906910384" oldPodUID="8cf317dc-8580-46c2-83ca-a079fdf8609e" podUID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2"
Jul 19 15:37:02.658805 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:02.658767    2572 generic.go:356] "Generic (PLEG): container finished" podID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" containerID="e31ca295b94dc8df47e27b13f003704ff10542240dac667c8766d9ffe9fc8d17" exitCode=0
Jul 19 15:37:02.659246 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:02.658851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906910384" event={"ID":"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2","Type":"ContainerDied","Data":"e31ca295b94dc8df47e27b13f003704ff10542240dac667c8766d9ffe9fc8d17"}
Jul 19 15:37:02.786464 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:02.786419    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf317dc-8580-46c2-83ca-a079fdf8609e" path="/var/lib/kubelet/pods/8cf317dc-8580-46c2-83ca-a079fdf8609e/volumes"
Jul 19 15:37:03.808905 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.808872    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:03.827873 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.827827    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7555-140401906910384_8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2/test-curl-7555-140401906910384/0.log"
Jul 19 15:37:03.853787 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.853749    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:03.855302 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.855279    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbgbn\" (UniqueName: \"kubernetes.io/projected/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2-kube-api-access-zbgbn\") pod \"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2\" (UID: \"8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2\") "
Jul 19 15:37:03.857596 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.857571    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2-kube-api-access-zbgbn" (OuterVolumeSpecName: "kube-api-access-zbgbn") pod "8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" (UID: "8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2"). InnerVolumeSpecName "kube-api-access-zbgbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:03.857818 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.857797    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7555-140401906910384"]
Jul 19 15:37:03.956766 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.956676    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbgbn\" (UniqueName: \"kubernetes.io/projected/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2-kube-api-access-zbgbn\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:03.978992 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.978941    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:03.979386 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.979368    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" containerName="test-curl-7555-140401906910384"
Jul 19 15:37:03.979386 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.979385    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" containerName="test-curl-7555-140401906910384"
Jul 19 15:37:03.979512 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.979460    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" containerName="test-curl-7555-140401906910384"
Jul 19 15:37:03.982482 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.982466    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:03.986647 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:03.986621    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:04.057310 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.057277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2qfw\" (UniqueName: \"kubernetes.io/projected/12696d04-7594-4749-92e5-5b2e29c3681a-kube-api-access-m2qfw\") pod \"test-curl-7555-140401906911392\" (UID: \"12696d04-7594-4749-92e5-5b2e29c3681a\") " pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:04.158885 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.158845    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2qfw\" (UniqueName: \"kubernetes.io/projected/12696d04-7594-4749-92e5-5b2e29c3681a-kube-api-access-m2qfw\") pod \"test-curl-7555-140401906911392\" (UID: \"12696d04-7594-4749-92e5-5b2e29c3681a\") " pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:04.167757 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.167727    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2qfw\" (UniqueName: \"kubernetes.io/projected/12696d04-7594-4749-92e5-5b2e29c3681a-kube-api-access-m2qfw\") pod \"test-curl-7555-140401906911392\" (UID: \"12696d04-7594-4749-92e5-5b2e29c3681a\") " pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:04.294009 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.293923    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:04.455652 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.455627    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:04.457676 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:37:04.457651    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12696d04_7594_4749_92e5_5b2e29c3681a.slice/crio-cdadb29b6d54411470984f334bf9c0e264f7b85bfd6d5475136c93da8dd6fa5e WatchSource:0}: Error finding container cdadb29b6d54411470984f334bf9c0e264f7b85bfd6d5475136c93da8dd6fa5e: Status 404 returned error can't find the container with id cdadb29b6d54411470984f334bf9c0e264f7b85bfd6d5475136c93da8dd6fa5e
Jul 19 15:37:04.671087 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.671054    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:04.671295 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.671098    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8abbf981b9aedbe1f6fc6a40382e9081c6d65cbd1e37a3ea6beeb0631869e4a8"
Jul 19 15:37:04.672438 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.672409    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906911392" event={"ID":"12696d04-7594-4749-92e5-5b2e29c3681a","Type":"ContainerStarted","Data":"cdadb29b6d54411470984f334bf9c0e264f7b85bfd6d5475136c93da8dd6fa5e"}
Jul 19 15:37:04.682053 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:37:04.682024    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7555-140401906910384\" is forbidden: User \"system:node:ip-10-0-135-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-98.ec2.internal' and this object" podUID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" pod="opendatahub/test-curl-7555-140401906910384"
Jul 19 15:37:04.787795 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:04.787761    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2" path="/var/lib/kubelet/pods/8db7f5ef-0b89-414b-8ba5-a5b70a88e4b2/volumes"
Jul 19 15:37:05.074918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.074815    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4" podUID="7bdca071-ef93-4317-a687-0fdb2e0665ba" containerName="main" containerID="cri-o://a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead" gracePeriod=45
Jul 19 15:37:05.336788 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.336763    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:37:05.373328 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373293    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-model-cache\") pod \"7bdca071-ef93-4317-a687-0fdb2e0665ba\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") "
Jul 19 15:37:05.373530 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373378    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs4x6\" (UniqueName: \"kubernetes.io/projected/7bdca071-ef93-4317-a687-0fdb2e0665ba-kube-api-access-fs4x6\") pod \"7bdca071-ef93-4317-a687-0fdb2e0665ba\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") "
Jul 19 15:37:05.373530 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373412    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-dshm\") pod \"7bdca071-ef93-4317-a687-0fdb2e0665ba\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") "
Jul 19 15:37:05.373530 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373428    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-tmp-dir\") pod \"7bdca071-ef93-4317-a687-0fdb2e0665ba\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") "
Jul 19 15:37:05.373530 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373515    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-home\") pod \"7bdca071-ef93-4317-a687-0fdb2e0665ba\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") "
Jul 19 15:37:05.373729 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373549    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdca071-ef93-4317-a687-0fdb2e0665ba-tls-certs\") pod \"7bdca071-ef93-4317-a687-0fdb2e0665ba\" (UID: \"7bdca071-ef93-4317-a687-0fdb2e0665ba\") "
Jul 19 15:37:05.373729 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373640    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-model-cache" (OuterVolumeSpecName: "model-cache") pod "7bdca071-ef93-4317-a687-0fdb2e0665ba" (UID: "7bdca071-ef93-4317-a687-0fdb2e0665ba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:05.373827 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373773    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-home" (OuterVolumeSpecName: "home") pod "7bdca071-ef93-4317-a687-0fdb2e0665ba" (UID: "7bdca071-ef93-4317-a687-0fdb2e0665ba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:05.373827 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373800    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7bdca071-ef93-4317-a687-0fdb2e0665ba" (UID: "7bdca071-ef93-4317-a687-0fdb2e0665ba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:05.373928 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373902    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:05.373928 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373923    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:05.373992 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.373938    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:05.375867 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.375845    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7bdca071-ef93-4317-a687-0fdb2e0665ba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7bdca071-ef93-4317-a687-0fdb2e0665ba" (UID: "7bdca071-ef93-4317-a687-0fdb2e0665ba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:37:05.375957 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.375927    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bdca071-ef93-4317-a687-0fdb2e0665ba-kube-api-access-fs4x6" (OuterVolumeSpecName: "kube-api-access-fs4x6") pod "7bdca071-ef93-4317-a687-0fdb2e0665ba" (UID: "7bdca071-ef93-4317-a687-0fdb2e0665ba"). InnerVolumeSpecName "kube-api-access-fs4x6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:05.376283 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.376258    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-dshm" (OuterVolumeSpecName: "dshm") pod "7bdca071-ef93-4317-a687-0fdb2e0665ba" (UID: "7bdca071-ef93-4317-a687-0fdb2e0665ba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:05.475052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.475010    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7bdca071-ef93-4317-a687-0fdb2e0665ba-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:05.475052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.475042    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs4x6\" (UniqueName: \"kubernetes.io/projected/7bdca071-ef93-4317-a687-0fdb2e0665ba-kube-api-access-fs4x6\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:05.475052 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.475053    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7bdca071-ef93-4317-a687-0fdb2e0665ba-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:05.679265 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.679170    2572 generic.go:356] "Generic (PLEG): container finished" podID="7bdca071-ef93-4317-a687-0fdb2e0665ba" containerID="a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead" exitCode=0
Jul 19 15:37:05.679265 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.679221    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"
Jul 19 15:37:05.679555 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.679257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4" event={"ID":"7bdca071-ef93-4317-a687-0fdb2e0665ba","Type":"ContainerDied","Data":"a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead"}
Jul 19 15:37:05.679555 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.679299    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4" event={"ID":"7bdca071-ef93-4317-a687-0fdb2e0665ba","Type":"ContainerDied","Data":"08a7f8a817561e75aff4aad98a7f8a9c6d366e27b72505445cfa69bd2cb30aca"}
Jul 19 15:37:05.679555 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.679321    2572 scope.go:117] "RemoveContainer" containerID="a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead"
Jul 19 15:37:05.680971 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.680949    2572 generic.go:356] "Generic (PLEG): container finished" podID="12696d04-7594-4749-92e5-5b2e29c3681a" containerID="f50452bb9045fd1009cdf92306068c3e3a1f77ce866b17f465187e2cc03560f8" exitCode=0
Jul 19 15:37:05.681089 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.680993    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906911392" event={"ID":"12696d04-7594-4749-92e5-5b2e29c3681a","Type":"ContainerDied","Data":"f50452bb9045fd1009cdf92306068c3e3a1f77ce866b17f465187e2cc03560f8"}
Jul 19 15:37:05.688719 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.688703    2572 scope.go:117] "RemoveContainer" containerID="a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead"
Jul 19 15:37:05.688969 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:37:05.688954    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead\": container with ID starting with a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead not found: ID does not exist" containerID="a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead"
Jul 19 15:37:05.689018 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.688977    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead"} err="failed to get container status \"a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead\": rpc error: code = NotFound desc = could not find container \"a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead\": container with ID starting with a26531a3f0ab1a2a412f6b725cdb5ba3a907495b764a2ae841cff03ed2cddead not found: ID does not exist"
Jul 19 15:37:05.709861 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.709831    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"]
Jul 19 15:37:05.713668 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.713646    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4bc178/rate-test-model-4bc178-kserve-659684dcff-cwld4"]
Jul 19 15:37:05.893064 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:05.893019    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk" podUID="18e6b54f-9a14-4505-8869-1c5e01069720" containerName="main" containerID="cri-o://1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be" gracePeriod=45
Jul 19 15:37:06.152430 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.152403    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:37:06.281940 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.281847    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e6b54f-9a14-4505-8869-1c5e01069720-tls-certs\") pod \"18e6b54f-9a14-4505-8869-1c5e01069720\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") "
Jul 19 15:37:06.282105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.281948    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-tmp-dir\") pod \"18e6b54f-9a14-4505-8869-1c5e01069720\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") "
Jul 19 15:37:06.282105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.281977    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-home\") pod \"18e6b54f-9a14-4505-8869-1c5e01069720\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") "
Jul 19 15:37:06.282105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282017    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-model-cache\") pod \"18e6b54f-9a14-4505-8869-1c5e01069720\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") "
Jul 19 15:37:06.282105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282043    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ntn4\" (UniqueName: \"kubernetes.io/projected/18e6b54f-9a14-4505-8869-1c5e01069720-kube-api-access-4ntn4\") pod \"18e6b54f-9a14-4505-8869-1c5e01069720\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") "
Jul 19 15:37:06.282105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282093    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-dshm\") pod \"18e6b54f-9a14-4505-8869-1c5e01069720\" (UID: \"18e6b54f-9a14-4505-8869-1c5e01069720\") "
Jul 19 15:37:06.282324 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282216    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18e6b54f-9a14-4505-8869-1c5e01069720" (UID: "18e6b54f-9a14-4505-8869-1c5e01069720"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:06.282324 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282239    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-home" (OuterVolumeSpecName: "home") pod "18e6b54f-9a14-4505-8869-1c5e01069720" (UID: "18e6b54f-9a14-4505-8869-1c5e01069720"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:06.282324 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282311    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-model-cache" (OuterVolumeSpecName: "model-cache") pod "18e6b54f-9a14-4505-8869-1c5e01069720" (UID: "18e6b54f-9a14-4505-8869-1c5e01069720"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:06.282524 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282462    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-tmp-dir\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.282524 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282476    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-home\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.282524 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.282486    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-model-cache\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.284250 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.284224    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e6b54f-9a14-4505-8869-1c5e01069720-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18e6b54f-9a14-4505-8869-1c5e01069720" (UID: "18e6b54f-9a14-4505-8869-1c5e01069720"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 19 15:37:06.284323 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.284308    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-dshm" (OuterVolumeSpecName: "dshm") pod "18e6b54f-9a14-4505-8869-1c5e01069720" (UID: "18e6b54f-9a14-4505-8869-1c5e01069720"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 19 15:37:06.284396 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.284312    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e6b54f-9a14-4505-8869-1c5e01069720-kube-api-access-4ntn4" (OuterVolumeSpecName: "kube-api-access-4ntn4") pod "18e6b54f-9a14-4505-8869-1c5e01069720" (UID: "18e6b54f-9a14-4505-8869-1c5e01069720"). InnerVolumeSpecName "kube-api-access-4ntn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:06.383107 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.383067    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ntn4\" (UniqueName: \"kubernetes.io/projected/18e6b54f-9a14-4505-8869-1c5e01069720-kube-api-access-4ntn4\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.383107 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.383099    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e6b54f-9a14-4505-8869-1c5e01069720-dshm\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.383107 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.383108    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e6b54f-9a14-4505-8869-1c5e01069720-tls-certs\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.687042 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.687009    2572 generic.go:356] "Generic (PLEG): container finished" podID="18e6b54f-9a14-4505-8869-1c5e01069720" containerID="1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be" exitCode=0
Jul 19 15:37:06.687253 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.687076    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"
Jul 19 15:37:06.687253 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.687107    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk" event={"ID":"18e6b54f-9a14-4505-8869-1c5e01069720","Type":"ContainerDied","Data":"1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be"}
Jul 19 15:37:06.687253 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.687156    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk" event={"ID":"18e6b54f-9a14-4505-8869-1c5e01069720","Type":"ContainerDied","Data":"4f71672f2179e68dcd152f9b7853483040076a24c1f4b414c93d3411bc31fa07"}
Jul 19 15:37:06.687253 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.687180    2572 scope.go:117] "RemoveContainer" containerID="1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be"
Jul 19 15:37:06.697141 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.697116    2572 scope.go:117] "RemoveContainer" containerID="1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be"
Jul 19 15:37:06.697493 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:37:06.697465    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be\": container with ID starting with 1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be not found: ID does not exist" containerID="1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be"
Jul 19 15:37:06.697579 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.697504    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be"} err="failed to get container status \"1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be\": rpc error: code = NotFound desc = could not find container \"1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be\": container with ID starting with 1acbcff33e561d0f4f8993885993e18afd1a7e487baaf50f7bc24548bacc68be not found: ID does not exist"
Jul 19 15:37:06.709248 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.709216    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"]
Jul 19 15:37:06.714565 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.714538    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ccd7ab/rate-test-model-ccd7ab-kserve-8658d6f655-hfkbk"]
Jul 19 15:37:06.792352 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.792310    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e6b54f-9a14-4505-8869-1c5e01069720" path="/var/lib/kubelet/pods/18e6b54f-9a14-4505-8869-1c5e01069720/volumes"
Jul 19 15:37:06.792728 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.792714    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bdca071-ef93-4317-a687-0fdb2e0665ba" path="/var/lib/kubelet/pods/7bdca071-ef93-4317-a687-0fdb2e0665ba/volumes"
Jul 19 15:37:06.827859 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.827829    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:06.846789 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.846760    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7555-140401906911392_12696d04-7594-4749-92e5-5b2e29c3681a/test-curl-7555-140401906911392/0.log"
Jul 19 15:37:06.868949 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.868908    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:06.873099 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.873066    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:06.886238 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.886208    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2qfw\" (UniqueName: \"kubernetes.io/projected/12696d04-7594-4749-92e5-5b2e29c3681a-kube-api-access-m2qfw\") pod \"12696d04-7594-4749-92e5-5b2e29c3681a\" (UID: \"12696d04-7594-4749-92e5-5b2e29c3681a\") "
Jul 19 15:37:06.888504 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.888477    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12696d04-7594-4749-92e5-5b2e29c3681a-kube-api-access-m2qfw" (OuterVolumeSpecName: "kube-api-access-m2qfw") pod "12696d04-7594-4749-92e5-5b2e29c3681a" (UID: "12696d04-7594-4749-92e5-5b2e29c3681a"). InnerVolumeSpecName "kube-api-access-m2qfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:06.987187 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.987096    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2qfw\" (UniqueName: \"kubernetes.io/projected/12696d04-7594-4749-92e5-5b2e29c3681a-kube-api-access-m2qfw\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:06.999314 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999275    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:06.999660 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999647    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bdca071-ef93-4317-a687-0fdb2e0665ba" containerName="main"
Jul 19 15:37:06.999712 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999662    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bdca071-ef93-4317-a687-0fdb2e0665ba" containerName="main"
Jul 19 15:37:06.999712 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999678    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12696d04-7594-4749-92e5-5b2e29c3681a" containerName="test-curl-7555-140401906911392"
Jul 19 15:37:06.999712 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999684    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="12696d04-7594-4749-92e5-5b2e29c3681a" containerName="test-curl-7555-140401906911392"
Jul 19 15:37:06.999712 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999699    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e6b54f-9a14-4505-8869-1c5e01069720" containerName="main"
Jul 19 15:37:06.999712 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999705    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e6b54f-9a14-4505-8869-1c5e01069720" containerName="main"
Jul 19 15:37:06.999877 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999757    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bdca071-ef93-4317-a687-0fdb2e0665ba" containerName="main"
Jul 19 15:37:06.999877 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999768    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e6b54f-9a14-4505-8869-1c5e01069720" containerName="main"
Jul 19 15:37:06.999877 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:06.999773    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="12696d04-7594-4749-92e5-5b2e29c3681a" containerName="test-curl-7555-140401906911392"
Jul 19 15:37:07.002806 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.002789    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:07.009549 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.009523    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:07.088245 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.088196    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvqp\" (UniqueName: \"kubernetes.io/projected/8bceea88-d878-495d-bd5d-3e266aa19516-kube-api-access-bgvqp\") pod \"test-curl-7555-140401906911392\" (UID: \"8bceea88-d878-495d-bd5d-3e266aa19516\") " pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:07.189853 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.189805    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvqp\" (UniqueName: \"kubernetes.io/projected/8bceea88-d878-495d-bd5d-3e266aa19516-kube-api-access-bgvqp\") pod \"test-curl-7555-140401906911392\" (UID: \"8bceea88-d878-495d-bd5d-3e266aa19516\") " pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:07.199145 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.199105    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvqp\" (UniqueName: \"kubernetes.io/projected/8bceea88-d878-495d-bd5d-3e266aa19516-kube-api-access-bgvqp\") pod \"test-curl-7555-140401906911392\" (UID: \"8bceea88-d878-495d-bd5d-3e266aa19516\") " pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:07.315638 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.315555    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:07.452106 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.452075    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:07.453718 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:37:07.453689    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bceea88_d878_495d_bd5d_3e266aa19516.slice/crio-b76afd56aafedd974efec813fedd1cf03258497089945886ffe713a3ae568451 WatchSource:0}: Error finding container b76afd56aafedd974efec813fedd1cf03258497089945886ffe713a3ae568451: Status 404 returned error can't find the container with id b76afd56aafedd974efec813fedd1cf03258497089945886ffe713a3ae568451
Jul 19 15:37:07.694845 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.694817    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:07.695041 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.694816    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdadb29b6d54411470984f334bf9c0e264f7b85bfd6d5475136c93da8dd6fa5e"
Jul 19 15:37:07.697044 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.697017    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906911392" event={"ID":"8bceea88-d878-495d-bd5d-3e266aa19516","Type":"ContainerStarted","Data":"b76afd56aafedd974efec813fedd1cf03258497089945886ffe713a3ae568451"}
Jul 19 15:37:07.706738 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:07.706712    2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7555-140401906911392" oldPodUID="12696d04-7594-4749-92e5-5b2e29c3681a" podUID="8bceea88-d878-495d-bd5d-3e266aa19516"
Jul 19 15:37:08.703773 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:08.703734    2572 generic.go:356] "Generic (PLEG): container finished" podID="8bceea88-d878-495d-bd5d-3e266aa19516" containerID="405d48fb5377620376f81ccc227dde08431179ad89e4e9c4ca37684e5d1ce514" exitCode=0
Jul 19 15:37:08.704246 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:08.703822    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7555-140401906911392" event={"ID":"8bceea88-d878-495d-bd5d-3e266aa19516","Type":"ContainerDied","Data":"405d48fb5377620376f81ccc227dde08431179ad89e4e9c4ca37684e5d1ce514"}
Jul 19 15:37:08.785829 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:08.785788    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12696d04-7594-4749-92e5-5b2e29c3681a" path="/var/lib/kubelet/pods/12696d04-7594-4749-92e5-5b2e29c3681a/volumes"
Jul 19 15:37:09.843555 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:09.843526    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:09.862562 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:09.862535    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7555-140401906911392_8bceea88-d878-495d-bd5d-3e266aa19516/test-curl-7555-140401906911392/0.log"
Jul 19 15:37:09.886406 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:09.886368    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:09.891981 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:09.891943    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7555-140401906911392"]
Jul 19 15:37:09.913005 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:09.912976    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgvqp\" (UniqueName: \"kubernetes.io/projected/8bceea88-d878-495d-bd5d-3e266aa19516-kube-api-access-bgvqp\") pod \"8bceea88-d878-495d-bd5d-3e266aa19516\" (UID: \"8bceea88-d878-495d-bd5d-3e266aa19516\") "
Jul 19 15:37:09.915394 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:09.915363    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bceea88-d878-495d-bd5d-3e266aa19516-kube-api-access-bgvqp" (OuterVolumeSpecName: "kube-api-access-bgvqp") pod "8bceea88-d878-495d-bd5d-3e266aa19516" (UID: "8bceea88-d878-495d-bd5d-3e266aa19516"). InnerVolumeSpecName "kube-api-access-bgvqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 19 15:37:10.014386 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.014194    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgvqp\" (UniqueName: \"kubernetes.io/projected/8bceea88-d878-495d-bd5d-3e266aa19516-kube-api-access-bgvqp\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:10.079618 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.079582    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"]
Jul 19 15:37:10.079944 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.079932    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bceea88-d878-495d-bd5d-3e266aa19516" containerName="test-curl-7555-140401906911392"
Jul 19 15:37:10.079987 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.079946    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bceea88-d878-495d-bd5d-3e266aa19516" containerName="test-curl-7555-140401906911392"
Jul 19 15:37:10.080030 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.080021    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bceea88-d878-495d-bd5d-3e266aa19516" containerName="test-curl-7555-140401906911392"
Jul 19 15:37:10.082992 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.082974    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:10.085597 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.085574    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:37:10.085597 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.085591    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 15:37:10.089899 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.089872    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"]
Jul 19 15:37:10.216166 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.216116    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea4853d-aa67-4e19-b0c8-244fb944956e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v\" (UID: \"0ea4853d-aa67-4e19-b0c8-244fb944956e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:10.317697 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.317601    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea4853d-aa67-4e19-b0c8-244fb944956e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v\" (UID: \"0ea4853d-aa67-4e19-b0c8-244fb944956e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:10.318230 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.318210    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea4853d-aa67-4e19-b0c8-244fb944956e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v\" (UID: \"0ea4853d-aa67-4e19-b0c8-244fb944956e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:10.394632 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.394578    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:10.532843 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.532813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"]
Jul 19 15:37:10.534840 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:37:10.534809    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea4853d_aa67_4e19_b0c8_244fb944956e.slice/crio-8ca07634da2e66635a46385c17dca62d78489a1da8b815d4cb64ee13d955b897 WatchSource:0}: Error finding container 8ca07634da2e66635a46385c17dca62d78489a1da8b815d4cb64ee13d955b897: Status 404 returned error can't find the container with id 8ca07634da2e66635a46385c17dca62d78489a1da8b815d4cb64ee13d955b897
Jul 19 15:37:10.715532 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.715503    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:10.715695 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.715545    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b76afd56aafedd974efec813fedd1cf03258497089945886ffe713a3ae568451"
Jul 19 15:37:10.716873 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.716848    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v" event={"ID":"0ea4853d-aa67-4e19-b0c8-244fb944956e","Type":"ContainerStarted","Data":"8ca07634da2e66635a46385c17dca62d78489a1da8b815d4cb64ee13d955b897"}
Jul 19 15:37:10.727204 ip-10-0-135-98 kubenswrapper[2572]: E0719 15:37:10.727176    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7555-140401906911392\" is forbidden: User \"system:node:ip-10-0-135-98.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-98.ec2.internal' and this object" podUID="8bceea88-d878-495d-bd5d-3e266aa19516" pod="opendatahub/test-curl-7555-140401906911392"
Jul 19 15:37:10.786510 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:10.786479    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bceea88-d878-495d-bd5d-3e266aa19516" path="/var/lib/kubelet/pods/8bceea88-d878-495d-bd5d-3e266aa19516/volumes"
Jul 19 15:37:11.724086 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:11.724053    2572 generic.go:356] "Generic (PLEG): container finished" podID="0ea4853d-aa67-4e19-b0c8-244fb944956e" containerID="a3be2c5c6ffb5e1cbd5db9c6b0a6f7393c8d352c60d75aab9a8c6f635a5e4d9d" exitCode=0
Jul 19 15:37:11.724505 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:11.724141    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v" event={"ID":"0ea4853d-aa67-4e19-b0c8-244fb944956e","Type":"ContainerDied","Data":"a3be2c5c6ffb5e1cbd5db9c6b0a6f7393c8d352c60d75aab9a8c6f635a5e4d9d"}
Jul 19 15:37:12.874839 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:12.874816    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:12.941215 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:12.941174    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea4853d-aa67-4e19-b0c8-244fb944956e-maas-api-service-ca\") pod \"0ea4853d-aa67-4e19-b0c8-244fb944956e\" (UID: \"0ea4853d-aa67-4e19-b0c8-244fb944956e\") "
Jul 19 15:37:12.941582 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:12.941562    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ea4853d-aa67-4e19-b0c8-244fb944956e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ea4853d-aa67-4e19-b0c8-244fb944956e" (UID: "0ea4853d-aa67-4e19-b0c8-244fb944956e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:37:13.042852 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:13.042756    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ea4853d-aa67-4e19-b0c8-244fb944956e-maas-api-service-ca\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:13.736162 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:13.736127    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"
Jul 19 15:37:13.736366 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:13.736157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v" event={"ID":"0ea4853d-aa67-4e19-b0c8-244fb944956e","Type":"ContainerDied","Data":"8ca07634da2e66635a46385c17dca62d78489a1da8b815d4cb64ee13d955b897"}
Jul 19 15:37:13.736366 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:13.736211    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8ca07634da2e66635a46385c17dca62d78489a1da8b815d4cb64ee13d955b897"
Jul 19 15:37:23.664071 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:23.664034    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:37:25.170293 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:25.170256    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"]
Jul 19 15:37:25.172526 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:25.172498    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4bc178-fz94v"]
Jul 19 15:37:26.786113 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:26.786080    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ea4853d-aa67-4e19-b0c8-244fb944956e" path="/var/lib/kubelet/pods/0ea4853d-aa67-4e19-b0c8-244fb944956e/volumes"
Jul 19 15:37:32.254222 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.254182    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"]
Jul 19 15:37:32.254773 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.254751    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ea4853d-aa67-4e19-b0c8-244fb944956e" containerName="revoke-keys"
Jul 19 15:37:32.254849 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.254777    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ea4853d-aa67-4e19-b0c8-244fb944956e" containerName="revoke-keys"
Jul 19 15:37:32.254907 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.254884    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ea4853d-aa67-4e19-b0c8-244fb944956e" containerName="revoke-keys"
Jul 19 15:37:32.260125 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.260104    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:32.262733 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.262702    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 19 15:37:32.262733 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.262726    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m8x6s\""
Jul 19 15:37:32.267706 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.267679    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"]
Jul 19 15:37:32.421627 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.421588    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff25170-6be9-45c7-8328-94caec6b0f79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb\" (UID: \"7ff25170-6be9-45c7-8328-94caec6b0f79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:32.523256 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.523153    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff25170-6be9-45c7-8328-94caec6b0f79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb\" (UID: \"7ff25170-6be9-45c7-8328-94caec6b0f79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:32.523906 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.523881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff25170-6be9-45c7-8328-94caec6b0f79-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb\" (UID: \"7ff25170-6be9-45c7-8328-94caec6b0f79\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:32.578362 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.578299    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:32.716015 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.715991    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"]
Jul 19 15:37:32.717317 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:37:32.717289    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff25170_6be9_45c7_8328_94caec6b0f79.slice/crio-96cac3c606cdd5d5aea89855f465e4f68c788f820e29658695dec9a880a27bb1 WatchSource:0}: Error finding container 96cac3c606cdd5d5aea89855f465e4f68c788f820e29658695dec9a880a27bb1: Status 404 returned error can't find the container with id 96cac3c606cdd5d5aea89855f465e4f68c788f820e29658695dec9a880a27bb1
Jul 19 15:37:32.842064 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:32.842029    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb" event={"ID":"7ff25170-6be9-45c7-8328-94caec6b0f79","Type":"ContainerStarted","Data":"96cac3c606cdd5d5aea89855f465e4f68c788f820e29658695dec9a880a27bb1"}
Jul 19 15:37:33.850080 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:33.850047    2572 generic.go:356] "Generic (PLEG): container finished" podID="7ff25170-6be9-45c7-8328-94caec6b0f79" containerID="3d60f1a58f30d5fb44f706131bf7bbe5cce3fab8fe8b9c2da3d8f7b1556e584e" exitCode=0
Jul 19 15:37:33.850512 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:33.850089    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb" event={"ID":"7ff25170-6be9-45c7-8328-94caec6b0f79","Type":"ContainerDied","Data":"3d60f1a58f30d5fb44f706131bf7bbe5cce3fab8fe8b9c2da3d8f7b1556e584e"}
Jul 19 15:37:34.987079 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:34.987048    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:35.146787 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:35.146700    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff25170-6be9-45c7-8328-94caec6b0f79-maas-api-service-ca\") pod \"7ff25170-6be9-45c7-8328-94caec6b0f79\" (UID: \"7ff25170-6be9-45c7-8328-94caec6b0f79\") "
Jul 19 15:37:35.147093 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:35.147071    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ff25170-6be9-45c7-8328-94caec6b0f79-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ff25170-6be9-45c7-8328-94caec6b0f79" (UID: "7ff25170-6be9-45c7-8328-94caec6b0f79"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 19 15:37:35.248081 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:35.248044    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff25170-6be9-45c7-8328-94caec6b0f79-maas-api-service-ca\") on node \"ip-10-0-135-98.ec2.internal\" DevicePath \"\""
Jul 19 15:37:35.862694 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:35.862665    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"
Jul 19 15:37:35.862876 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:35.862710    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb" event={"ID":"7ff25170-6be9-45c7-8328-94caec6b0f79","Type":"ContainerDied","Data":"96cac3c606cdd5d5aea89855f465e4f68c788f820e29658695dec9a880a27bb1"}
Jul 19 15:37:35.862876 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:35.862760    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96cac3c606cdd5d5aea89855f465e4f68c788f820e29658695dec9a880a27bb1"
Jul 19 15:37:45.659988 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:45.659945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4c2hn"]
Jul 19 15:37:47.340531 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:47.340498    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"]
Jul 19 15:37:47.351745 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:47.351717    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ccd7ab-dj6cb"]
Jul 19 15:37:48.785755 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:37:48.785717    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ff25170-6be9-45c7-8328-94caec6b0f79" path="/var/lib/kubelet/pods/7ff25170-6be9-45c7-8328-94caec6b0f79/volumes"
Jul 19 15:38:00.674152 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:00.674118    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7s6gp_54167599-0655-4ff1-90af-8b9e45cc8144/server/0.log"
Jul 19 15:38:00.800513 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:00.800486    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wxvrm_d868dfab-760d-4ac1-a740-f3169366f186/manager/0.log"
Jul 19 15:38:01.073427 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:01.073323    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-j92mz_75a1ecb9-85a5-401a-a97f-42952ce5a8ff/copy-manifests/0.log"
Jul 19 15:38:01.086621 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:01.086597    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-j92mz_75a1ecb9-85a5-401a-a97f-42952ce5a8ff/manager/0.log"
Jul 19 15:38:01.840462 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:01.840430    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29741250-zt5gv_077f3c67-4e55-4ee9-a270-7b404de6582a/cleanup/0.log"
Jul 19 15:38:02.746403 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:02.746375    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl_7cce55a3-59a3-441f-a612-695a2f21b3dc/util/0.log"
Jul 19 15:38:02.753646 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:02.753620    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl_7cce55a3-59a3-441f-a612-695a2f21b3dc/pull/0.log"
Jul 19 15:38:02.760959 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:02.760917    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl_7cce55a3-59a3-441f-a612-695a2f21b3dc/extract/0.log"
Jul 19 15:38:02.878429 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:02.878399    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t_6cba9242-4608-4b81-bdc2-c097c9a6a27e/util/0.log"
Jul 19 15:38:02.887380 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:02.887320    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t_6cba9242-4608-4b81-bdc2-c097c9a6a27e/pull/0.log"
Jul 19 15:38:02.894106 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:02.894083    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t_6cba9242-4608-4b81-bdc2-c097c9a6a27e/extract/0.log"
Jul 19 15:38:03.009929 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.009849    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr_a40290bb-3cf2-427c-a534-7e2c73d58c5d/pull/0.log"
Jul 19 15:38:03.016420 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.016396    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr_a40290bb-3cf2-427c-a534-7e2c73d58c5d/extract/0.log"
Jul 19 15:38:03.022304 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.022281    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr_a40290bb-3cf2-427c-a534-7e2c73d58c5d/util/0.log"
Jul 19 15:38:03.135752 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.135722    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l_2f014e40-21c6-4f61-88c9-4c3434236f75/pull/0.log"
Jul 19 15:38:03.142122 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.142097    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l_2f014e40-21c6-4f61-88c9-4c3434236f75/extract/0.log"
Jul 19 15:38:03.148010 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.147990    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l_2f014e40-21c6-4f61-88c9-4c3434236f75/util/0.log"
Jul 19 15:38:03.620823 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.620795    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-554nf_d9d6a028-4308-450d-bc2e-2a768c7bf88a/kuadrant-console-plugin/0.log"
Jul 19 15:38:03.989903 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:03.989814    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4c2hn_191e16a3-ae71-49f7-9aab-77293ebb28c1/limitador/0.log"
Jul 19 15:38:04.869201 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:04.869168    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-6csfm_212b794d-9204-48b0-a4f3-d2521b7b427d/kube-auth-proxy/0.log"
Jul 19 15:38:05.395909 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:05.395876    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c56786b4-222cc_590bf656-2b4e-473e-bbb0-9d73010f86ad/router/0.log"
Jul 19 15:38:06.336905 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:06.336871    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-f4pfn_a3aefd58-d7e5-49f6-94a6-4d199e88d7af/main/0.log"
Jul 19 15:38:12.337310 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.337274    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bxflm/must-gather-4j4w4"]
Jul 19 15:38:12.339686 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.337758    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ff25170-6be9-45c7-8328-94caec6b0f79" containerName="revoke-keys"
Jul 19 15:38:12.339686 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.337773    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ff25170-6be9-45c7-8328-94caec6b0f79" containerName="revoke-keys"
Jul 19 15:38:12.339686 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.337868    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ff25170-6be9-45c7-8328-94caec6b0f79" containerName="revoke-keys"
Jul 19 15:38:12.340918 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.340903    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.344093 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.344069    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bxflm\"/\"default-dockercfg-7krnv\""
Jul 19 15:38:12.344251 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.344156    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bxflm\"/\"openshift-service-ca.crt\""
Jul 19 15:38:12.344746 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.344728    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bxflm\"/\"kube-root-ca.crt\""
Jul 19 15:38:12.352427 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.352404    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bxflm/must-gather-4j4w4"]
Jul 19 15:38:12.469077 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.469030    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9ss\" (UniqueName: \"kubernetes.io/projected/fdf98327-e686-4502-ba6a-ab095de87f6c-kube-api-access-cc9ss\") pod \"must-gather-4j4w4\" (UID: \"fdf98327-e686-4502-ba6a-ab095de87f6c\") " pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.469265 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.469108    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fdf98327-e686-4502-ba6a-ab095de87f6c-must-gather-output\") pod \"must-gather-4j4w4\" (UID: \"fdf98327-e686-4502-ba6a-ab095de87f6c\") " pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.569916 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.569870    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9ss\" (UniqueName: \"kubernetes.io/projected/fdf98327-e686-4502-ba6a-ab095de87f6c-kube-api-access-cc9ss\") pod \"must-gather-4j4w4\" (UID: \"fdf98327-e686-4502-ba6a-ab095de87f6c\") " pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.570125 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.569931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fdf98327-e686-4502-ba6a-ab095de87f6c-must-gather-output\") pod \"must-gather-4j4w4\" (UID: \"fdf98327-e686-4502-ba6a-ab095de87f6c\") " pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.570269 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.570253    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fdf98327-e686-4502-ba6a-ab095de87f6c-must-gather-output\") pod \"must-gather-4j4w4\" (UID: \"fdf98327-e686-4502-ba6a-ab095de87f6c\") " pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.578597 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.578577    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9ss\" (UniqueName: \"kubernetes.io/projected/fdf98327-e686-4502-ba6a-ab095de87f6c-kube-api-access-cc9ss\") pod \"must-gather-4j4w4\" (UID: \"fdf98327-e686-4502-ba6a-ab095de87f6c\") " pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.650576 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.650487    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bxflm/must-gather-4j4w4"
Jul 19 15:38:12.796105 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:12.796079    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bxflm/must-gather-4j4w4"]
Jul 19 15:38:12.798398 ip-10-0-135-98 kubenswrapper[2572]: W0719 15:38:12.798366    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdf98327_e686_4502_ba6a_ab095de87f6c.slice/crio-a4b22f1e20f52759a7a381676aaf13792b40b30c9809c099e37ec7c6ace32054 WatchSource:0}: Error finding container a4b22f1e20f52759a7a381676aaf13792b40b30c9809c099e37ec7c6ace32054: Status 404 returned error can't find the container with id a4b22f1e20f52759a7a381676aaf13792b40b30c9809c099e37ec7c6ace32054
Jul 19 15:38:13.067677 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:13.067587    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/must-gather-4j4w4" event={"ID":"fdf98327-e686-4502-ba6a-ab095de87f6c","Type":"ContainerStarted","Data":"a4b22f1e20f52759a7a381676aaf13792b40b30c9809c099e37ec7c6ace32054"}
Jul 19 15:38:14.080640 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:14.080527    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/must-gather-4j4w4" event={"ID":"fdf98327-e686-4502-ba6a-ab095de87f6c","Type":"ContainerStarted","Data":"a6c78d92d404984d654d97163f7750ad7d5405da585af5d4dba822a02a2f7da3"}
Jul 19 15:38:14.080640 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:14.080579    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/must-gather-4j4w4" event={"ID":"fdf98327-e686-4502-ba6a-ab095de87f6c","Type":"ContainerStarted","Data":"da3d8ee2d2e115369a41ff6f217cabd547beae07fa96aaba99e15e1f5e97e9df"}
Jul 19 15:38:14.100405 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:14.100324    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bxflm/must-gather-4j4w4" podStartSLOduration=1.217840246 podStartE2EDuration="2.100301627s" totalImagesPullingTime="882.461381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-19 15:38:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:38:14.09927579 +0000 UTC m=+3095.800723845" watchObservedRunningTime="2026-07-19 15:38:14.100301627 +0000 UTC m=+3095.801749682"
Jul 19 15:38:15.225784 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:15.225749    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gwmb8_d965c61d-cbd0-4f16-bc14-7d87a547cbfe/global-pull-secret-syncer/0.log"
Jul 19 15:38:15.402361 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:15.402316    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8fx5w_b3dbd6e4-0932-4dfb-b293-9db3588d47c0/konnectivity-agent/0.log"
Jul 19 15:38:15.457573 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:15.457541    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-98.ec2.internal_52a2fdb785bd958c3f182967e497262f/haproxy/0.log"
Jul 19 15:38:20.210878 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.210743    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl_7cce55a3-59a3-441f-a612-695a2f21b3dc/extract/0.log"
Jul 19 15:38:20.233628 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.233598    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl_7cce55a3-59a3-441f-a612-695a2f21b3dc/util/0.log"
Jul 19 15:38:20.258368 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.258323    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rlcpl_7cce55a3-59a3-441f-a612-695a2f21b3dc/pull/0.log"
Jul 19 15:38:20.290957 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.290910    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t_6cba9242-4608-4b81-bdc2-c097c9a6a27e/extract/0.log"
Jul 19 15:38:20.318205 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.318164    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t_6cba9242-4608-4b81-bdc2-c097c9a6a27e/util/0.log"
Jul 19 15:38:20.341539 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.341326    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dlx5t_6cba9242-4608-4b81-bdc2-c097c9a6a27e/pull/0.log"
Jul 19 15:38:20.381347 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.381173    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr_a40290bb-3cf2-427c-a534-7e2c73d58c5d/extract/0.log"
Jul 19 15:38:20.417557 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.417526    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr_a40290bb-3cf2-427c-a534-7e2c73d58c5d/util/0.log"
Jul 19 15:38:20.449046 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.449005    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73pcmgr_a40290bb-3cf2-427c-a534-7e2c73d58c5d/pull/0.log"
Jul 19 15:38:20.482799 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.482709    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l_2f014e40-21c6-4f61-88c9-4c3434236f75/extract/0.log"
Jul 19 15:38:20.507182 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.507147    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l_2f014e40-21c6-4f61-88c9-4c3434236f75/util/0.log"
Jul 19 15:38:20.529088 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:20.529056    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1m6v9l_2f014e40-21c6-4f61-88c9-4c3434236f75/pull/0.log"
Jul 19 15:38:21.219493 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:21.219457    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-554nf_d9d6a028-4308-450d-bc2e-2a768c7bf88a/kuadrant-console-plugin/0.log"
Jul 19 15:38:21.490004 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:21.489917    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4c2hn_191e16a3-ae71-49f7-9aab-77293ebb28c1/limitador/0.log"
Jul 19 15:38:23.468458 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.468419    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-f2csq_b30b8fc1-74ae-409b-b60b-ddd88fd659bb/cluster-monitoring-operator/0.log"
Jul 19 15:38:23.496304 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.496274    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-q8g6l_ed8ce068-4b87-45cf-84aa-440fd4405a49/kube-state-metrics/0.log"
Jul 19 15:38:23.515057 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.515027    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-q8g6l_ed8ce068-4b87-45cf-84aa-440fd4405a49/kube-rbac-proxy-main/0.log"
Jul 19 15:38:23.537692 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.537668    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-q8g6l_ed8ce068-4b87-45cf-84aa-440fd4405a49/kube-rbac-proxy-self/0.log"
Jul 19 15:38:23.574264 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.574237    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79898cf4b8-csrkk_f8849838-b127-4594-8040-1e3345fef0e0/metrics-server/0.log"
Jul 19 15:38:23.602799 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.602738    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-8xtnp_c1f6f9ba-ec81-478d-83ae-eee4536efd47/monitoring-plugin/0.log"
Jul 19 15:38:23.812109 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.812016    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8q2g_8281eeb4-bee7-4399-83c8-0e02a863793b/node-exporter/0.log"
Jul 19 15:38:23.833221 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.833186    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8q2g_8281eeb4-bee7-4399-83c8-0e02a863793b/kube-rbac-proxy/0.log"
Jul 19 15:38:23.857938 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.857909    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q8q2g_8281eeb4-bee7-4399-83c8-0e02a863793b/init-textfile/0.log"
Jul 19 15:38:23.884914 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.884884    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-n7ngq_8eef126c-9281-452b-b1f6-f81bd1fba198/kube-rbac-proxy-main/0.log"
Jul 19 15:38:23.907507 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.907473    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-n7ngq_8eef126c-9281-452b-b1f6-f81bd1fba198/kube-rbac-proxy-self/0.log"
Jul 19 15:38:23.928959 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:23.928934    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-n7ngq_8eef126c-9281-452b-b1f6-f81bd1fba198/openshift-state-metrics/0.log"
Jul 19 15:38:24.327457 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:24.327422    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7656454fbb-qwdr6_cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b/thanos-query/0.log"
Jul 19 15:38:24.348881 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:24.348849    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7656454fbb-qwdr6_cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b/kube-rbac-proxy-web/0.log"
Jul 19 15:38:24.371607 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:24.371558    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7656454fbb-qwdr6_cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b/kube-rbac-proxy/0.log"
Jul 19 15:38:24.394649 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:24.394619    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7656454fbb-qwdr6_cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b/prom-label-proxy/0.log"
Jul 19 15:38:24.416962 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:24.416935    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7656454fbb-qwdr6_cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b/kube-rbac-proxy-rules/0.log"
Jul 19 15:38:24.437996 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:24.437962    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7656454fbb-qwdr6_cd3dfa90-bdd8-46b8-818c-deb07a6cdb2b/kube-rbac-proxy-metrics/0.log"
Jul 19 15:38:25.824642 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:25.824604    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"]
Jul 19 15:38:25.834387 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:25.834324    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:25.837026 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:25.836995    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"]
Jul 19 15:38:26.009432 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.009384    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-sys\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.009645 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.009459    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrkdn\" (UniqueName: \"kubernetes.io/projected/77200e12-da30-47dd-ba13-2c1eb87c5349-kube-api-access-lrkdn\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.009645 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.009518    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-podres\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.009645 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.009609    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-proc\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.009645 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.009643    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-lib-modules\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.110180 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.110144    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-proc\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.110471 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.110450    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-lib-modules\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.110643 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.110630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-sys\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.110746 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.110733    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrkdn\" (UniqueName: \"kubernetes.io/projected/77200e12-da30-47dd-ba13-2c1eb87c5349-kube-api-access-lrkdn\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.110854 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.110842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-podres\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.111205 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.111188    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-proc\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.111457 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.111441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-lib-modules\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.111596 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.111582    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-sys\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.112100 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.112079    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/77200e12-da30-47dd-ba13-2c1eb87c5349-podres\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.121378 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.121355    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrkdn\" (UniqueName: \"kubernetes.io/projected/77200e12-da30-47dd-ba13-2c1eb87c5349-kube-api-access-lrkdn\") pod \"perf-node-gather-daemonset-ng6xh\" (UID: \"77200e12-da30-47dd-ba13-2c1eb87c5349\") " pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.148928 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.148895    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:26.386037 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:26.386002    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"]
Jul 19 15:38:27.191922 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:27.191879    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh" event={"ID":"77200e12-da30-47dd-ba13-2c1eb87c5349","Type":"ContainerStarted","Data":"feb244a359ef6e0e347bd9a2f425dac8db408d81a8b71672924246ac14b7e6e6"}
Jul 19 15:38:27.192569 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:27.192531    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:27.192698 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:27.192581    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh" event={"ID":"77200e12-da30-47dd-ba13-2c1eb87c5349","Type":"ContainerStarted","Data":"5900000bdfc2c077220d70b725b2dc806fd6b4190fb4330564783ad15b078b7b"}
Jul 19 15:38:27.212486 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:27.212426    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh" podStartSLOduration=2.212404937 podStartE2EDuration="2.212404937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-19 15:38:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-19 15:38:27.2093748 +0000 UTC m=+3108.910822857" watchObservedRunningTime="2026-07-19 15:38:27.212404937 +0000 UTC m=+3108.913852994"
Jul 19 15:38:28.525002 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:28.524963    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzq6p_926fed00-bf12-4e4d-a042-9811b951d1fc/dns/0.log"
Jul 19 15:38:28.550244 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:28.550213    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hzq6p_926fed00-bf12-4e4d-a042-9811b951d1fc/kube-rbac-proxy/0.log"
Jul 19 15:38:28.625582 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:28.625559    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cchfz_388682d3-450c-43ac-8c2d-448bf2d98b9d/dns-node-resolver/0.log"
Jul 19 15:38:29.202879 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:29.202844    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-66c5bfdfc4-8bwrm_86cc9cad-cdd1-45fc-857d-d5961be30d0c/registry/0.log"
Jul 19 15:38:29.311997 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:29.311953    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n9985_a092a032-b5e0-4112-8106-b4201294fc7b/node-ca/0.log"
Jul 19 15:38:30.511563 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:30.511536    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-6csfm_212b794d-9204-48b0-a4f3-d2521b7b427d/kube-auth-proxy/0.log"
Jul 19 15:38:30.686462 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:30.686422    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c56786b4-222cc_590bf656-2b4e-473e-bbb0-9d73010f86ad/router/0.log"
Jul 19 15:38:31.380667 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:31.380636    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-24nxl_d3f0b971-6e80-4db5-8008-77916f998dfe/serve-healthcheck-canary/0.log"
Jul 19 15:38:31.936862 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:31.936833    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xwv7_81b36509-ec8b-4c2a-a0ac-165c8c817a7b/kube-rbac-proxy/0.log"
Jul 19 15:38:31.959092 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:31.959065    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xwv7_81b36509-ec8b-4c2a-a0ac-165c8c817a7b/exporter/0.log"
Jul 19 15:38:31.980897 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:31.980869    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6xwv7_81b36509-ec8b-4c2a-a0ac-165c8c817a7b/extractor/0.log"
Jul 19 15:38:33.210615 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:33.210585    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bxflm/perf-node-gather-daemonset-ng6xh"
Jul 19 15:38:34.902759 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:34.902721    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-7s6gp_54167599-0655-4ff1-90af-8b9e45cc8144/server/0.log"
Jul 19 15:38:35.004758 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:35.004726    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-wxvrm_d868dfab-760d-4ac1-a740-f3169366f186/manager/0.log"
Jul 19 15:38:35.178014 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:35.177981    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-j92mz_75a1ecb9-85a5-401a-a97f-42952ce5a8ff/manager/0.log"
Jul 19 15:38:35.202693 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:35.202659    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c7885d785-j92mz_75a1ecb9-85a5-401a-a97f-42952ce5a8ff/copy-manifests/0.log"
Jul 19 15:38:36.597971 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:36.597935    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5cc4ccf5bd-jdqc8_30a0cbe3-dbee-4f57-bb83-7d2d704aead2/manager/0.log"
Jul 19 15:38:36.602653 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:36.602629    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-6csfm_212b794d-9204-48b0-a4f3-d2521b7b427d/kube-auth-proxy/0.log"
Jul 19 15:38:36.741843 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:36.741813    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c56786b4-222cc_590bf656-2b4e-473e-bbb0-9d73010f86ad/router/0.log"
Jul 19 15:38:38.165312 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:38.165231    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ccf86bb4b-6csfm_212b794d-9204-48b0-a4f3-d2521b7b427d/kube-auth-proxy/0.log"
Jul 19 15:38:38.311521 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:38.311480    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c56786b4-222cc_590bf656-2b4e-473e-bbb0-9d73010f86ad/router/0.log"
Jul 19 15:38:43.157285 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.157256    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/kube-multus-additional-cni-plugins/0.log"
Jul 19 15:38:43.180672 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.180639    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/egress-router-binary-copy/0.log"
Jul 19 15:38:43.205806 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.205777    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/cni-plugins/0.log"
Jul 19 15:38:43.231514 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.231481    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/bond-cni-plugin/0.log"
Jul 19 15:38:43.254142 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.254108    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/routeoverride-cni/0.log"
Jul 19 15:38:43.280639 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.280603    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/whereabouts-cni-bincopy/0.log"
Jul 19 15:38:43.310810 ip-10-0-135-98 kubenswrapper[2572]: I0719 15:38:43.310779    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-4wwcf_de5d621a-6d73-4131-a323-f001e1cf14c0/whereabouts-cni/0.log"