Jul 26 15:36:20.755528 ip-10-0-138-109 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 26 15:36:20.755542 ip-10-0-138-109 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 26 15:36:20.755551 ip-10-0-138-109 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 26 15:36:20.755874 ip-10-0-138-109 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 26 15:36:30.881806 ip-10-0-138-109 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 26 15:36:30.881825 ip-10-0-138-109 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e1638830c439490da1435f532f670892 --
Jul 26 15:38:19.140399 ip-10-0-138-109 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 15:38:19.444046 ip-10-0-138-109 kubenswrapper[2534]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:38:19.444046 ip-10-0-138-109 kubenswrapper[2534]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 15:38:19.444046 ip-10-0-138-109 kubenswrapper[2534]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:38:19.444046 ip-10-0-138-109 kubenswrapper[2534]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 15:38:19.444046 ip-10-0-138-109 kubenswrapper[2534]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 15:38:19.445456 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.445314    2534 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 15:38:19.449495 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449479    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:19.449495 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449496    2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449500    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449503    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449507    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449510    2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449513    2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449516    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449519    2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449522    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449525    2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449527    2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449530    2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449533    2534 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449536    2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449539    2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449541    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449544    2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449547    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449551    2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449554    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:19.449559 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449556    2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449559    2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449561    2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449565    2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449568    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449571    2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449574    2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449577    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449580    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449583    2534 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449585    2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449588    2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449590    2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449593    2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449596    2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449598    2534 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449601    2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449604    2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449607    2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:19.450064 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449612    2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449616    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449619    2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449622    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449625    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449628    2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449631    2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449633    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449636    2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449638    2534 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449641    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449643    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449646    2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449648    2534 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449651    2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449655    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449658    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449661    2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449664    2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449666    2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:19.450538 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449670    2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449672    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449675    2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449678    2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449681    2534 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449683    2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449686    2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449688    2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449691    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449694    2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449697    2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449702    2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449706    2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449710    2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449715    2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449718    2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449721    2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449724    2534 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449726    2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:19.451016 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449729    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449732    2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449735    2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449738    2534 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449741    2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449743    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449746    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449750    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449752    2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449755    2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449758    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449761    2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449763    2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449766    2534 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449768    2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449771    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449774    2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449776    2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449779    2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.449781    2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:19.451488 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450510    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450516    2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450519    2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450522    2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450525    2534 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450534    2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450536    2534 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450539    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450542    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450545    2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450548    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450551    2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450554    2534 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450556    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450561    2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450564    2534 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450568    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450571    2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450574    2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450578    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:19.451965 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450580    2534 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450583    2534 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450585    2534 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450588    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450591    2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450593    2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450596    2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450598    2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450601    2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450604    2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450606    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450609    2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450611    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450614    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450617    2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450619    2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450622    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450624    2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450627    2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450631    2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:19.452507 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450635    2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450638    2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450640    2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450643    2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450646    2534 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450648    2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450651    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450653    2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450657    2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450659    2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450662    2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450665    2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450668    2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450671    2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450674    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450677    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450679    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450682    2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450685    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:19.452998 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450687    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450690    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450692    2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450695    2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450697    2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450700    2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450702    2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450705    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450707    2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450710    2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450713    2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450715    2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450718    2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450721    2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450724    2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450727    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450730    2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450733    2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450736    2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450738    2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:19.453481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450741    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450743    2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450746    2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450749    2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450753    2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450757    2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450760    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450763    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450766    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450769    2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450772    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450775    2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450778    2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450781    2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450784    2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450786    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450789    2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450791    2534 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450794    2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:19.453985 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.450797    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451177    2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451183    2534 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451186    2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451189    2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451192    2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451195    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451198    2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451202    2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451205    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451207    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451210    2534 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451213    2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451215    2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451218    2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451221    2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451223    2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451226    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451228    2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451231    2534 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451234    2534 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:19.454468 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451236    2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451239    2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451243    2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451247    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451250    2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451253    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451257    2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451260    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451263    2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451266    2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451269    2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451272    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451274    2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451277    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451279    2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451282    2534 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451285    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451289    2534 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451292    2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:19.454974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451295    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451298    2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451300    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451302    2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451305    2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451308    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451310    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451313    2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451315    2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451318    2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451321    2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451323    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451326    2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451328    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451331    2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451333    2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451336    2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451339    2534 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451341    2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:19.455449 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451344    2534 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451347    2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451349    2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451354    2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451358    2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451360    2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451363    2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451366    2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451368    2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451373    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451376    2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451378    2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451384    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451387    2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451390    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451393    2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451395    2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451398    2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451400    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:19.455898 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451403    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451405    2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451408    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451410    2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451413    2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451415    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451418    2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451420    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451423    2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451425    2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451428    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451430    2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451433    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451435    2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451438    2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451441    2534 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451443    2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451446    2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451448    2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451450    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:19.456372 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451453    2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.451455    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451538    2534 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451548    2534 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451564    2534 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451569    2534 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451575    2534 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451579    2534 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451583    2534 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451587    2534 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451590    2534 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451594    2534 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451598    2534 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451601    2534 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451604    2534 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451608    2534 flags.go:64] FLAG: --cgroup-root=""
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451611    2534 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451614    2534 flags.go:64] FLAG: --client-ca-file=""
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451617    2534 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451619    2534 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451624    2534 flags.go:64] FLAG: --cluster-domain=""
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451627    2534 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451630    2534 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451633    2534 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 15:38:19.456857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451636    2534 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451640    2534 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451643    2534 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451646    2534 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451650    2534 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451653    2534 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451656    2534 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451659    2534 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451662    2534 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451665    2534 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451671    2534 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451674    2534 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451677    2534 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451680    2534 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451685    2534 flags.go:64] FLAG: --enable-server="true"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451689    2534 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451693    2534 flags.go:64] FLAG: --event-burst="100"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451696    2534 flags.go:64] FLAG: --event-qps="50"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451699    2534 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451702    2534 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451706    2534 flags.go:64] FLAG: --eviction-hard=""
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451709    2534 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451712    2534 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451716    2534 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451719    2534 flags.go:64] FLAG: --eviction-soft=""
Jul 26 15:38:19.457590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451722    2534 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451724    2534 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451727    2534 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451731    2534 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451734    2534 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451737    2534 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451741    2534 flags.go:64] FLAG: --feature-gates=""
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451744    2534 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451748    2534 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451751    2534 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451755    2534 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451758    2534 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451761    2534 flags.go:64] FLAG: --help="false"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451764    2534 flags.go:64] FLAG: --hostname-override="ip-10-0-138-109.ec2.internal"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451767    2534 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451770    2534 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451773    2534 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451778    2534 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451781    2534 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451784    2534 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451787    2534 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451791    2534 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451794    2534 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 15:38:19.458255 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451797    2534 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451801    2534 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451804    2534 flags.go:64] FLAG: --kube-reserved=""
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451807    2534 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451809    2534 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451813    2534 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451815    2534 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451818    2534 flags.go:64] FLAG: --lock-file=""
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451821    2534 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451824    2534 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451827    2534 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451832    2534 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451835    2534 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451838    2534 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451841    2534 flags.go:64] FLAG: --logging-format="text"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451844    2534 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451848    2534 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451851    2534 flags.go:64] FLAG: --manifest-url=""
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451854    2534 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451858    2534 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451861    2534 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451865    2534 flags.go:64] FLAG: --max-pods="110"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451868    2534 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451871    2534 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451874    2534 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 15:38:19.458832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451877    2534 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451881    2534 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451884    2534 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451887    2534 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451894    2534 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451899    2534 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451902    2534 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451905    2534 flags.go:64] FLAG: --pod-cidr=""
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451908    2534 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451914    2534 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451917    2534 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451920    2534 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451923    2534 flags.go:64] FLAG: --port="10250"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451926    2534 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451929    2534 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00a9d41db131697b7"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451932    2534 flags.go:64] FLAG: --qos-reserved=""
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451935    2534 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451939    2534 flags.go:64] FLAG: --register-node="true"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451942    2534 flags.go:64] FLAG: --register-with-taints=""
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451946    2534 flags.go:64] FLAG: --registry-burst="10"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451949    2534 flags.go:64] FLAG: --registry-qps="5"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451951    2534 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451954    2534 flags.go:64] FLAG: --reserved-memory=""
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451965    2534 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451968    2534 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 15:38:19.459444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451972    2534 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451975    2534 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451978    2534 flags.go:64] FLAG: --runonce="false"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451981    2534 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451985    2534 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451987    2534 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451990    2534 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451993    2534 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.451997    2534 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452000    2534 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452003    2534 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452006    2534 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452011    2534 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452014    2534 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452017    2534 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452020    2534 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452031    2534 flags.go:64] FLAG: --system-cgroups=""
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452034    2534 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452039    2534 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452042    2534 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452045    2534 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452052    2534 flags.go:64] FLAG: --tls-min-version=""
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452055    2534 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452058    2534 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452061    2534 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 15:38:19.460066 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452064    2534 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452067    2534 flags.go:64] FLAG: --v="2"
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452071    2534 flags.go:64] FLAG: --version="false"
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452075    2534 flags.go:64] FLAG: --vmodule=""
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452079    2534 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452082    2534 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452210    2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452214    2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452219    2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452223    2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452227    2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452231    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452234    2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452237    2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452240    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452243    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452246    2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452249    2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452252    2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452256    2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:19.460701 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452259    2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452262    2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452264    2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452267    2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452270    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452273    2534 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452276    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452279    2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452282    2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452285    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452287    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452290    2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452293    2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452295    2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452298    2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452301    2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452303    2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452306    2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452309    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:19.461360 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452311    2534 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452314    2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452317    2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452320    2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452323    2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452325    2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452328    2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452330    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452333    2534 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452336    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452338    2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452341    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452345    2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452348    2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452350    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452353    2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452357    2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452360    2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452364    2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452367    2534 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:19.462063 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452370    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452373    2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452375    2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452378    2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452381    2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452384    2534 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452386    2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452389    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452391    2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452394    2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452396    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452399    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452401    2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452404    2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452407    2534 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452409    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452412    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452414    2534 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452417    2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452419    2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452422    2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:19.462584 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452425    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452427    2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452430    2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452434    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452437    2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452439    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452442    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452445    2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452447    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452450    2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452459    2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452461    2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452464    2534 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452466    2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452469    2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452471    2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452474    2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452476    2534 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452479    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452481    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:19.463098 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452484    2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452486    2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452489    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452491    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.452494    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.452506    2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.459140    2534 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.459155    2534 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.459715    2534 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.459722    2534 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459794    2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459800    2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459804    2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459807    2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459811    2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:19.463605 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459813    2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459816    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459819    2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459823    2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459826    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459829    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459832    2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459841    2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459844    2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459847    2534 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459849    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459852    2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459855    2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459858    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459860    2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459863    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459866    2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459868    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459871    2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459874    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:19.463974 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459877    2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459880    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459882    2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459885    2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459889    2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459891    2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459894    2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459896    2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459899    2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459902    2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459905    2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459907    2534 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459910    2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459913    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459915    2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459918    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459921    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459929    2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459932    2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459934    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:19.464489 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459942    2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459946    2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459948    2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459951    2534 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459954    2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459956    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459959    2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459962    2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459966    2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459970    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459974    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459976    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459979    2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459983    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459985    2534 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459988    2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459991    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459994    2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.459997    2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:19.464977 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460001    2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460005    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460008    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460011    2534 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460014    2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460016    2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460019    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460021    2534 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460024    2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460027    2534 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460029    2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460032    2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460034    2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460037    2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460045    2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460048    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460051    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460054    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460056    2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460058    2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:19.465534 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460062    2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460065    2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460068    2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460072    2534 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460075    2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460078    2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460081    2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460084    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460087    2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460091    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460094    2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460097    2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460100    2534 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460103    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:19.466025 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460121    2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.460128    2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460279    2534 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460284    2534 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460287    2534 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460290    2534 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460293    2534 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460295    2534 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460298    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460300    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460303    2534 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460307    2534 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460311    2534 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460321    2534 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460324    2534 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 15:38:19.466393 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460327    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460330    2534 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460332    2534 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460335    2534 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460337    2534 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460340    2534 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460343    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460345    2534 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460348    2534 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460350    2534 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460353    2534 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460356    2534 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460359    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460361    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460364    2534 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460367    2534 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460369    2534 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460372    2534 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460374    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460377    2534 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 15:38:19.466776 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460379    2534 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460382    2534 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460385    2534 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460387    2534 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460389    2534 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460392    2534 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460394    2534 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460397    2534 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460400    2534 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460403    2534 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460406    2534 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460413    2534 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460416    2534 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460419    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460421    2534 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460424    2534 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460426    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460429    2534 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460431    2534 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460434    2534 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 15:38:19.467335 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460436    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460439    2534 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460441    2534 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460444    2534 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460446    2534 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460449    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460452    2534 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460454    2534 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460457    2534 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460460    2534 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460462    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460465    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460468    2534 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460470    2534 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460473    2534 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460475    2534 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460478    2534 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460480    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460483    2534 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460485    2534 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 15:38:19.467827 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460488    2534 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460490    2534 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460493    2534 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460495    2534 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460503    2534 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460506    2534 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460508    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460511    2534 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460514    2534 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460516    2534 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460518    2534 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460521    2534 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460525    2534 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460527    2534 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460530    2534 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460532    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460535    2534 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460538    2534 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460541    2534 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 15:38:19.468455 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460543    2534 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460546    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460549    2534 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460552    2534 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460556    2534 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460559    2534 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:19.460562    2534 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.460567    2534 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.460671    2534 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.464162    2534 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.465219    2534 server.go:1019] "Starting client certificate rotation"
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.465315    2534 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.465928    2534 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 15:38:19.468915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.467201    2534 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 15:38:19.469273 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.469130    2534 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 15:38:19.477852 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.477836    2534 log.go:25] "Validated CRI v1 runtime API"
Jul 26 15:38:19.481782 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.481767    2534 log.go:25] "Validated CRI v1 image API"
Jul 26 15:38:19.482642 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.482621    2534 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 15:38:19.486142 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.486123    2534 fs.go:135] Filesystem UUIDs: map[3e2f4943-e3be-417b-a2f2-78719be1b98b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 909358b9-e24b-481e-99a2-4660d4eb5437:/dev/nvme0n1p4]
Jul 26 15:38:19.486217 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.486142    2534 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 15:38:19.491803 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.491786    2534 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 15:38:19.491865 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.491702    2534 manager.go:216] Machine: {Timestamp:2026-07-26 15:38:19.490600532 +0000 UTC m=+0.259791287 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3048446 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f3b2012ad5a09d0515c517df4021e SystemUUID:ec2f3b20-12ad-5a09-d051-5c517df4021e BootID:e1638830-c439-490d-a143-5f532f670892 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:28:b2:77:2d:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:28:b2:77:2d:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a6:24:af:69:44:5d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 15:38:19.491865 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.491849    2534 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 15:38:19.491968 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.491936    2534 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 15:38:19.494144 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.494096    2534 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 15:38:19.494329 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.494149    2534 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-109.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 15:38:19.494802 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.494790    2534 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 15:38:19.494855 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.494808    2534 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 15:38:19.494855 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.494818    2534 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 15:38:19.495319 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.495307    2534 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 15:38:19.496117 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.496094    2534 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 15:38:19.498750 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498737    2534 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 15:38:19.498885 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498875    2534 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 15:38:19.498956 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498920    2534 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 15:38:19.498956 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498935    2534 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 15:38:19.498956 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498949    2534 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 15:38:19.499086 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498963    2534 kubelet.go:398] "Adding apiserver pod source"
Jul 26 15:38:19.499086 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.498974    2534 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 15:38:19.499478 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.499464    2534 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 15:38:19.499544 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.499489    2534 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 15:38:19.502409 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.502379    2534 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 15:38:19.504083 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.504066    2534 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 15:38:19.504159 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.504097    2534 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 15:38:19.505357 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505343    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505368    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505380    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505385    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505395    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505400    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505409    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 15:38:19.505417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505414    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 15:38:19.505589 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505421    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 15:38:19.505589 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505427    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 15:38:19.505589 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505435    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 15:38:19.505589 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505440    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 15:38:19.505984 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505975    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 15:38:19.506016 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.505986    2534 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 15:38:19.509009 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.508995    2534 server.go:1268] "Started kubelet"
Jul 26 15:38:19.509128 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.509088    2534 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 15:38:19.509262 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.509191    2534 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 15:38:19.509413 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.509281    2534 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 15:38:19.509811 ip-10-0-138-109 systemd[1]: Started Kubernetes Kubelet.
Jul 26 15:38:19.510991 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.510863    2534 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 15:38:19.511235 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.511222    2534 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 15:38:19.512355 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.512330    2534 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-109.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 15:38:19.512452 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.512361    2534 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-109.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 15:38:19.512452 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.512373    2534 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 15:38:19.513992 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.513974    2534 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 15:38:19.514849 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.514151    2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-109.ec2.internal.18c5e1131bc7b653  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-109.ec2.internal,UID:ip-10-0-138-109.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-109.ec2.internal,},FirstTimestamp:2026-07-26 15:38:19.508971091 +0000 UTC m=+0.278161847,LastTimestamp:2026-07-26 15:38:19.508971091 +0000 UTC m=+0.278161847,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-109.ec2.internal,}"
Jul 26 15:38:19.517029 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517012    2534 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 15:38:19.517029 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517020    2534 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 15:38:19.517175 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.517131    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:19.517566 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517543    2534 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 15:38:19.517649 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517547    2534 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 15:38:19.517649 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517628    2534 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 15:38:19.517825 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517812    2534 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 15:38:19.517825 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.517826    2534 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 15:38:19.518067 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.518049    2534 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cs2qd"
Jul 26 15:38:19.518904 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.518888    2534 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 15:38:19.518904 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.518904    2534 factory.go:55] Registering systemd factory
Jul 26 15:38:19.519025 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.518914    2534 factory.go:223] Registration of the systemd container factory successfully
Jul 26 15:38:19.519156 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.519143    2534 factory.go:153] Registering CRI-O factory
Jul 26 15:38:19.519156 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.519157    2534 factory.go:223] Registration of the crio container factory successfully
Jul 26 15:38:19.519250 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.519174    2534 factory.go:103] Registering Raw factory
Jul 26 15:38:19.519250 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.519184    2534 manager.go:1196] Started watching for new ooms in manager
Jul 26 15:38:19.519614 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.519602    2534 manager.go:349] Starting recovery of all containers
Jul 26 15:38:19.523845 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.523815    2534 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-109.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 15:38:19.523945 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.523922    2534 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 15:38:19.524069 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.524051    2534 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cs2qd"
Jul 26 15:38:19.531667 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.531429    2534 manager.go:354] Recovery completed
Jul 26 15:38:19.535955 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.535942    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.537782 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.537764    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.537878 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.537799    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.537878 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.537820    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.538374 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.538360    2534 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 15:38:19.538436 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.538375    2534 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 15:38:19.538436 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.538393    2534 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 15:38:19.538436 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.538367    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.539045 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.539032    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.539157 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.539055    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.539157 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.539064    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.540249 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.540189    2534 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-109.ec2.internal.18c5e1131d7f537a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-109.ec2.internal,UID:ip-10-0-138-109.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-109.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-109.ec2.internal,},FirstTimestamp:2026-07-26 15:38:19.537781626 +0000 UTC m=+0.306972385,LastTimestamp:2026-07-26 15:38:19.537781626 +0000 UTC m=+0.306972385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-109.ec2.internal,}"
Jul 26 15:38:19.541392 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.541378    2534 policy_none.go:49] "None policy: Start"
Jul 26 15:38:19.541454 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.541399    2534 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 15:38:19.541454 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.541413    2534 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 15:38:19.543359 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.543350    2534 policy_none.go:47] "Start"
Jul 26 15:38:19.577797 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.577654    2534 manager.go:337] "Starting Device Plugin manager"
Jul 26 15:38:19.577882 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.577820    2534 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 15:38:19.577882 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.577836    2534 server.go:87] "Starting device plugin registration server"
Jul 26 15:38:19.578034 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.578024    2534 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 15:38:19.578075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.578036    2534 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 15:38:19.578158 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.578139    2534 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 15:38:19.578258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.578215    2534 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 15:38:19.578258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.578222    2534 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 15:38:19.578702 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.578687    2534 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 15:38:19.578813 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.578728    2534 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:19.678326 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.678304    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.680071 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.679237    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.680071 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.679265    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.680071 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.679276    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.680071 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.679307    2534 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-109.ec2.internal"
Jul 26 15:38:19.686751 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.686736    2534 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-109.ec2.internal"
Jul 26 15:38:19.686853 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.686761    2534 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-109.ec2.internal\": node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:19.700970 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.700915    2534 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 15:38:19.702283 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.702262    2534 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 15:38:19.702364 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.702298    2534 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 15:38:19.702364 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.702323    2534 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 15:38:19.702364 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.702354    2534 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 15:38:19.705332 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.705317    2534 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:19.716036 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.716012    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:19.803086 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.803064    2534 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"]
Jul 26 15:38:19.803178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.803134    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.803981 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.803966    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.804048 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.803995    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.804048 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.804005    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.806046 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806034    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.806165 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806151    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:19.806206 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806180    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.806814 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806798    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.806906 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806821    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.806906 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806830    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.806906 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806850    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.806906 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806873    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.806906 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.806886    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.808839 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.808824    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:19.808908 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.808848    2534 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 15:38:19.809489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.809470    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 15:38:19.809569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.809500    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 15:38:19.809569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:19.809513    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeHasSufficientPID"
Jul 26 15:38:19.816298 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.816280    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:19.826720 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.826697    2534 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-109.ec2.internal\" not found" node="ip-10-0-138-109.ec2.internal"
Jul 26 15:38:19.830308 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:19.830292    2534 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-109.ec2.internal\" not found" node="ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.502086 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:21.502063    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:21.504322 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.504306    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c0b441876693b98f22616c38ca505a9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal\" (UID: \"c0b441876693b98f22616c38ca505a9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.504405 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.504331    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0b441876693b98f22616c38ca505a9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal\" (UID: \"c0b441876693b98f22616c38ca505a9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.504405 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.504349    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5682356d1d816bbe296f83312dcd1ed-config\") pod \"kube-apiserver-proxy-ip-10-0-138-109.ec2.internal\" (UID: \"a5682356d1d816bbe296f83312dcd1ed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.603046 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:21.602971    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:21.605168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.605152    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c0b441876693b98f22616c38ca505a9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal\" (UID: \"c0b441876693b98f22616c38ca505a9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.605220 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.605176    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0b441876693b98f22616c38ca505a9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal\" (UID: \"c0b441876693b98f22616c38ca505a9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.605220 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.605194    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5682356d1d816bbe296f83312dcd1ed-config\") pod \"kube-apiserver-proxy-ip-10-0-138-109.ec2.internal\" (UID: \"a5682356d1d816bbe296f83312dcd1ed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.605296 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.605213    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c0b441876693b98f22616c38ca505a9b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal\" (UID: \"c0b441876693b98f22616c38ca505a9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.605296 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.605223    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5682356d1d816bbe296f83312dcd1ed-config\") pod \"kube-apiserver-proxy-ip-10-0-138-109.ec2.internal\" (UID: \"a5682356d1d816bbe296f83312dcd1ed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.605296 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.605224    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0b441876693b98f22616c38ca505a9b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal\" (UID: \"c0b441876693b98f22616c38ca505a9b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.703374 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:21.703346    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:21.714701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.714680    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.718088 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.718066    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:21.804209 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:21.804176    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:21.904709 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:21.904636    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:21.920927 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:21.920910    2534 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:22.004855 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.004824    2534 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-109.ec2.internal\" not found"
Jul 26 15:38:22.050262 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.050223    2534 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 15:38:22.050661 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.050395    2534 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 15:38:22.050661 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.050413    2534 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 15:38:22.076346 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.076325    2534 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:22.084897 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.084880    2534 apiserver.go:52] "Watching apiserver"
Jul 26 15:38:22.092261 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.092244    2534 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 15:38:22.094286 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.093430    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-z6jld","openshift-image-registry/node-ca-d7l7j","openshift-multus/multus-additional-cni-plugins-6g9f9","openshift-multus/multus-n4z5c","openshift-multus/network-metrics-daemon-tfwln","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb","openshift-dns/node-resolver-9szlv","openshift-network-diagnostics/network-check-target-zkvdb","openshift-network-operator/iptables-alerter-9fdhf","openshift-ovn-kubernetes/ovnkube-node-2gbw2","kube-system/global-pull-secret-syncer-f5rzd","kube-system/konnectivity-agent-t4kd8"]
Jul 26 15:38:22.095819 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.095800    2534 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:22.096336 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.096309    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.098271 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.098257    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.099716 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.099668    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.099716 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.099708    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.099839 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.099745    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7k9gr\""
Jul 26 15:38:22.100252 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.100240    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.101525 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.101505    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 15:38:22.101597 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.101505    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.101873 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.101856    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.101925 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.101896    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w6m7w\""
Jul 26 15:38:22.102259 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102226    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.102259 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102227    2534 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 15:38:22.102447 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102431    2534 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:22.102571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102550    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 15:38:22.102642 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102614    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 15:38:22.102701 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102685    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.102752 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102720    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 15:38:22.102801 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.102776    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tf5pw\""
Jul 26 15:38:22.103032 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.103017    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.104307 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.104290    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.104399 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.104358    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 15:38:22.104459 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.104352    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:22.104726 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.104711    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wp755\""
Jul 26 15:38:22.106475 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.106459    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.107447 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107409    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-host\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107447 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107433    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9de5f64f-1327-445c-bd32-2314e29b690a-tmp\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107447 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107448    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-cnibin\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.107624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107470    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-cnibin\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.107624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107498    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-etc-kubernetes\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.107624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107527    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ddzd\" (UniqueName: \"kubernetes.io/projected/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-kube-api-access-7ddzd\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.107624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107552    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysctl-d\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107581    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-systemd\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107611    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9de5f64f-1327-445c-bd32-2314e29b690a-etc-tuned\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107629    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbtt7\" (UniqueName: \"kubernetes.io/projected/5b3996da-64e2-474e-b2c7-1b255286357b-kube-api-access-wbtt7\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.107796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107660    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz44m\" (UniqueName: \"kubernetes.io/projected/d33720d3-9a94-4ecf-9a95-e427352f11b1-kube-api-access-wz44m\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.107796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107686    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4vjh\" (UniqueName: \"kubernetes.io/projected/0590d803-5bd9-4739-abf3-13c8bdacc2c0-kube-api-access-m4vjh\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.107796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107711    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-sys\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107741    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-k8s-cni-cncf-io\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.107796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107769    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-daemon-config\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107798    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-run\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107822    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b3996da-64e2-474e-b2c7-1b255286357b-host\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107848    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107872    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-cni-multus\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107898    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysconfig\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107919    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b3996da-64e2-474e-b2c7-1b255286357b-serviceca\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107935    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-os-release\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.107964 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107954    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-cni-bin\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107971    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-lib-modules\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107986    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-system-cni-dir\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.107999    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-conf-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108015    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108045    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-modprobe-d\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108064    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysctl-conf\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108079    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xt8c\" (UniqueName: \"kubernetes.io/projected/9de5f64f-1327-445c-bd32-2314e29b690a-kube-api-access-6xt8c\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108101    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108136    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-cni-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108163    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-socket-dir-parent\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108179    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-kubelet\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108193    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108207    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108255    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-system-cni-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108276    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-os-release\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108296    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-cni-binary-copy\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108309    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-netns\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108345    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-hostroot\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108363    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-multus-certs\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108379    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-kubernetes\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108392    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-var-lib-kubelet\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.108664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108522    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.109114 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.108977    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.109114 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.109049    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tfj84\""
Jul 26 15:38:22.109114 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.109076    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.109114 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.109107    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 15:38:22.110677 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.110658    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:22.110773 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.110713    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:22.110773 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.110736    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.110881 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.110792    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.110881 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.110798    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kdph2\""
Jul 26 15:38:22.113013 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.112816    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.113512 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.113494    2534 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 15:38:22.113596 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.113555    2534 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"
Jul 26 15:38:22.114947 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.114932    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 15:38:22.115123 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.115107    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.115570 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.115553    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.115656 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.115586    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.115656 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.115609    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cksct\""
Jul 26 15:38:22.115762 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.115711    2534 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 15:38:22.117072 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.117031    2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 15:33:21 +0000 UTC" deadline="2028-03-22 15:53:18.861016841 +0000 UTC"
Jul 26 15:38:22.117072 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.117071    2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14520h14m56.743950536s"
Jul 26 15:38:22.117188 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.117074    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 15:38:22.117188 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.117101    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-prt8g\""
Jul 26 15:38:22.117384 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.117368    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.117952 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.117848    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:22.118646 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.118621    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 15:38:22.118646 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.118640    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 15:38:22.118780 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.118662    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 15:38:22.119091 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.119072    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 15:38:22.119284 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.119266    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 15:38:22.120494 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.120468    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal"]
Jul 26 15:38:22.120715 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.120700    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.121173 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.121152    2534 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 15:38:22.121311 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.121295    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal"]
Jul 26 15:38:22.123296 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.123277    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 15:38:22.123477 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.123464    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 15:38:22.123546 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.123518    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-28cnb\""
Jul 26 15:38:22.203459 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.203437    2534 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 15:38:22.206996 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.206980    2534 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r9x28"
Jul 26 15:38:22.208526 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208506    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b3996da-64e2-474e-b2c7-1b255286357b-host\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.208607 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208533    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.208607 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208552    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-cni-multus\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.208607 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208570    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.208718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208620    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5b3996da-64e2-474e-b2c7-1b255286357b-host\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.208718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208628    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-cni-multus\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.208718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208657    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:22.208718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208676    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.208718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208694    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.208718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208712    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysconfig\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208730    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-sys-fs\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208754    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-ovnkube-config\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208770    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh7lf\" (UniqueName: \"kubernetes.io/projected/98395466-d07f-467b-b216-c46e5f2a9884-kube-api-access-dh7lf\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208777    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysconfig\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208794    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-system-cni-dir\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208826    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-tmp-dir\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208830    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-system-cni-dir\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208852    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-run-netns\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.208915 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208892    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98395466-d07f-467b-b216-c46e5f2a9884-ovn-node-metrics-cert\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208925    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysctl-conf\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.208975    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xt8c\" (UniqueName: \"kubernetes.io/projected/9de5f64f-1327-445c-bd32-2314e29b690a-kube-api-access-6xt8c\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209001    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209002    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysctl-conf\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209043    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209099    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-socket-dir-parent\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209154    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-ovn\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209178    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209200    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-os-release\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209220    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-cni-binary-copy\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209257    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-socket-dir-parent\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209253    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-netns\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209312    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-netns\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209321    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-os-release\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209361    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjp2\" (UniqueName: \"kubernetes.io/projected/ec31e446-d355-4bcd-b047-145753788e89-kube-api-access-qzjp2\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209387    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-kubernetes\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209427    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-host\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209446    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-cnibin\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209463    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-cnibin\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209446    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-kubernetes\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209486    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51eb5653-f3da-4900-8f1d-790a4ec88c7d-kubelet-config\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209511    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-host\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209516    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-cnibin\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209511    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51eb5653-f3da-4900-8f1d-790a4ec88c7d-dbus\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209529    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-cnibin\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209556    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysctl-d\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209616    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbtt7\" (UniqueName: \"kubernetes.io/projected/5b3996da-64e2-474e-b2c7-1b255286357b-kube-api-access-wbtt7\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209654    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-cni-bin\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209635    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-sysctl-d\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209693    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-sys\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.209988 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209714    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-k8s-cni-cncf-io\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209736    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-systemd-units\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209761    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-run\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209762    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-k8s-cni-cncf-io\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209769    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-sys\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209785    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ss9\" (UniqueName: \"kubernetes.io/projected/fdcd463c-c172-449b-9271-a3f2f30a133b-kube-api-access-g2ss9\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209808    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-kubelet\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209816    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-run\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209822    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b3996da-64e2-474e-b2c7-1b255286357b-serviceca\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209838    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-os-release\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209835    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-cni-binary-copy\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209884    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-cni-bin\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209888    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209904    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40b953aa-b2b6-4ac4-a36b-94d60df89a5f-konnectivity-ca\") pod \"konnectivity-agent-t4kd8\" (UID: \"40b953aa-b2b6-4ac4-a36b-94d60df89a5f\") " pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209926    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-lib-modules\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209919    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-os-release\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.209937    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-cni-bin\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.210758 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210011    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-conf-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210035    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210064    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-device-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210088    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fdcd463c-c172-449b-9271-a3f2f30a133b-host-slash\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210109    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-lib-modules\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210120    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-conf-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.210149    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210174    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-etc-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210209    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-cni-netd\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.210246    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:22.710208803 +0000 UTC m=+1.894274941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210283    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-modprobe-d\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210298    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5b3996da-64e2-474e-b2c7-1b255286357b-serviceca\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210302    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-cni-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210346    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-kubelet\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210380    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-hostroot\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210390    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-cni-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210396    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-node-log\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.211489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210416    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-env-overrides\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210434    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-modprobe-d\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210434    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-var-lib-kubelet\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210474    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-hostroot\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210489    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210506    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210521    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-system-cni-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210536    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-multus-certs\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210549    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-etc-kubernetes\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210563    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ddzd\" (UniqueName: \"kubernetes.io/projected/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-kube-api-access-7ddzd\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210586    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-socket-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210598    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-host-run-multus-certs\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210619    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-system-cni-dir\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210633    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-etc-kubernetes\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210662    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-registration-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210688    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d33720d3-9a94-4ecf-9a95-e427352f11b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210703    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-var-lib-kubelet\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210758    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9de5f64f-1327-445c-bd32-2314e29b690a-tmp\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210778    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-var-lib-kubelet\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210794    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fdcd463c-c172-449b-9271-a3f2f30a133b-iptables-alerter-script\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210821    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-hosts-file\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210844    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-systemd\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210873    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-var-lib-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210892    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-ovnkube-script-lib\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210908    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-systemd\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210928    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9de5f64f-1327-445c-bd32-2314e29b690a-etc-tuned\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210942    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9de5f64f-1327-445c-bd32-2314e29b690a-etc-systemd\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210947    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz44m\" (UniqueName: \"kubernetes.io/projected/d33720d3-9a94-4ecf-9a95-e427352f11b1-kube-api-access-wz44m\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210982    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d33720d3-9a94-4ecf-9a95-e427352f11b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.210975    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4vjh\" (UniqueName: \"kubernetes.io/projected/0590d803-5bd9-4739-abf3-13c8bdacc2c0-kube-api-access-m4vjh\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211018    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40b953aa-b2b6-4ac4-a36b-94d60df89a5f-agent-certs\") pod \"konnectivity-agent-t4kd8\" (UID: \"40b953aa-b2b6-4ac4-a36b-94d60df89a5f\") " pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211016    2534 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211041    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-slash\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211064    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-log-socket\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.212561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211102    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-daemon-config\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.213011 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211124    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.213011 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211148    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.213011 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211190    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52767\" (UniqueName: \"kubernetes.io/projected/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-kube-api-access-52767\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.213011 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.211494    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-multus-daemon-config\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.213506 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.213492    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9de5f64f-1327-445c-bd32-2314e29b690a-tmp\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.213546 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.213526    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9de5f64f-1327-445c-bd32-2314e29b690a-etc-tuned\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.214379 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.214364    2534 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r9x28"
Jul 26 15:38:22.221472 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.221449    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xt8c\" (UniqueName: \"kubernetes.io/projected/9de5f64f-1327-445c-bd32-2314e29b690a-kube-api-access-6xt8c\") pod \"tuned-z6jld\" (UID: \"9de5f64f-1327-445c-bd32-2314e29b690a\") " pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.223071 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.223044    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbtt7\" (UniqueName: \"kubernetes.io/projected/5b3996da-64e2-474e-b2c7-1b255286357b-kube-api-access-wbtt7\") pod \"node-ca-d7l7j\" (UID: \"5b3996da-64e2-474e-b2c7-1b255286357b\") " pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.225190 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.225168    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ddzd\" (UniqueName: \"kubernetes.io/projected/93b1d160-11ca-47b7-a1bc-1f36cceb9ce7-kube-api-access-7ddzd\") pod \"multus-n4z5c\" (UID: \"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7\") " pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.225283 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.225201    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4vjh\" (UniqueName: \"kubernetes.io/projected/0590d803-5bd9-4739-abf3-13c8bdacc2c0-kube-api-access-m4vjh\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.225465 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.225445    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz44m\" (UniqueName: \"kubernetes.io/projected/d33720d3-9a94-4ecf-9a95-e427352f11b1-kube-api-access-wz44m\") pod \"multus-additional-cni-plugins-6g9f9\" (UID: \"d33720d3-9a94-4ecf-9a95-e427352f11b1\") " pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.312305 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312280    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40b953aa-b2b6-4ac4-a36b-94d60df89a5f-konnectivity-ca\") pod \"konnectivity-agent-t4kd8\" (UID: \"40b953aa-b2b6-4ac4-a36b-94d60df89a5f\") " pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312316    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-device-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312332    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fdcd463c-c172-449b-9271-a3f2f30a133b-host-slash\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312380    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-device-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312403    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-etc-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312418    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-cni-netd\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312434    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-node-log\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312449    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-env-overrides\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312458    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fdcd463c-c172-449b-9271-a3f2f30a133b-host-slash\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312470    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-etc-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312490 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312470    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-socket-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312490    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-cni-netd\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312506    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-node-log\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312525    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-registration-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312556    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-socket-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312556    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fdcd463c-c172-449b-9271-a3f2f30a133b-iptables-alerter-script\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312593    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-hosts-file\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312619    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-systemd\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312628    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-registration-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312642    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-hosts-file\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312641    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-var-lib-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312685    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-ovnkube-script-lib\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312700    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-systemd\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312722    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40b953aa-b2b6-4ac4-a36b-94d60df89a5f-agent-certs\") pod \"konnectivity-agent-t4kd8\" (UID: \"40b953aa-b2b6-4ac4-a36b-94d60df89a5f\") " pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312747    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-slash\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312747    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-var-lib-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312817    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-slash\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.312975 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312833    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/40b953aa-b2b6-4ac4-a36b-94d60df89a5f-konnectivity-ca\") pod \"konnectivity-agent-t4kd8\" (UID: \"40b953aa-b2b6-4ac4-a36b-94d60df89a5f\") " pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312934    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-env-overrides\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312941    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-log-socket\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312986    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.312988    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-log-socket\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313013    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313059    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-etc-selinux\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313066    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52767\" (UniqueName: \"kubernetes.io/projected/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-kube-api-access-52767\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313096    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313121    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.313131    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313144    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313169    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.313191    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:38:22.813171014 +0000 UTC m=+1.997237132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313170    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fdcd463c-c172-449b-9271-a3f2f30a133b-iptables-alerter-script\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313214    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.313790 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313217    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-ovnkube-script-lib\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313207    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313222    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-sys-fs\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313280    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-openvswitch\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313298    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ec31e446-d355-4bcd-b047-145753788e89-sys-fs\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313327    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-ovnkube-config\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313367    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh7lf\" (UniqueName: \"kubernetes.io/projected/98395466-d07f-467b-b216-c46e5f2a9884-kube-api-access-dh7lf\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313404    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-tmp-dir\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313431    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-run-netns\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313479    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98395466-d07f-467b-b216-c46e5f2a9884-ovn-node-metrics-cert\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313507    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-ovn\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313520    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-run-netns\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313533    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313568    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjp2\" (UniqueName: \"kubernetes.io/projected/ec31e446-d355-4bcd-b047-145753788e89-kube-api-access-qzjp2\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313599    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51eb5653-f3da-4900-8f1d-790a4ec88c7d-kubelet-config\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313603    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-run-ovn\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313570    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314385 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313623    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51eb5653-f3da-4900-8f1d-790a4ec88c7d-dbus\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313650    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-cni-bin\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313659    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/51eb5653-f3da-4900-8f1d-790a4ec88c7d-kubelet-config\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313676    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-systemd-units\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313700    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ss9\" (UniqueName: \"kubernetes.io/projected/fdcd463c-c172-449b-9271-a3f2f30a133b-kube-api-access-g2ss9\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313727    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/51eb5653-f3da-4900-8f1d-790a4ec88c7d-dbus\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313732    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-tmp-dir\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313729    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-kubelet\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313767    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-kubelet\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313768    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-systemd-units\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313778    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/98395466-d07f-467b-b216-c46e5f2a9884-ovnkube-config\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.314894 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.313795    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98395466-d07f-467b-b216-c46e5f2a9884-host-cni-bin\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.315339 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.315320    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/40b953aa-b2b6-4ac4-a36b-94d60df89a5f-agent-certs\") pod \"konnectivity-agent-t4kd8\" (UID: \"40b953aa-b2b6-4ac4-a36b-94d60df89a5f\") " pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.315491 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.315476    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/98395466-d07f-467b-b216-c46e5f2a9884-ovn-node-metrics-cert\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.319207 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.319192    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:22.319274 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.319212    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:22.319274 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.319222    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:22.319356 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.319339    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:38:22.819318984 +0000 UTC m=+2.003385117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:22.321458 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.321438    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ss9\" (UniqueName: \"kubernetes.io/projected/fdcd463c-c172-449b-9271-a3f2f30a133b-kube-api-access-g2ss9\") pod \"iptables-alerter-9fdhf\" (UID: \"fdcd463c-c172-449b-9271-a3f2f30a133b\") " pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.321590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.321574    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52767\" (UniqueName: \"kubernetes.io/projected/d998ee4d-b750-48d8-8fbd-2e086b48e1c5-kube-api-access-52767\") pod \"node-resolver-9szlv\" (UID: \"d998ee4d-b750-48d8-8fbd-2e086b48e1c5\") " pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.321687 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.321669    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh7lf\" (UniqueName: \"kubernetes.io/projected/98395466-d07f-467b-b216-c46e5f2a9884-kube-api-access-dh7lf\") pod \"ovnkube-node-2gbw2\" (UID: \"98395466-d07f-467b-b216-c46e5f2a9884\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.321970 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.321955    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjp2\" (UniqueName: \"kubernetes.io/projected/ec31e446-d355-4bcd-b047-145753788e89-kube-api-access-qzjp2\") pod \"aws-ebs-csi-driver-node-9mkfb\" (UID: \"ec31e446-d355-4bcd-b047-145753788e89\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.357010 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.356977    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5682356d1d816bbe296f83312dcd1ed.slice/crio-55ebc754fa57861da8250d71e6a482d6718faf3efc5badb2b2bd1bdb289e6af8 WatchSource:0}: Error finding container 55ebc754fa57861da8250d71e6a482d6718faf3efc5badb2b2bd1bdb289e6af8: Status 404 returned error can't find the container with id 55ebc754fa57861da8250d71e6a482d6718faf3efc5badb2b2bd1bdb289e6af8
Jul 26 15:38:22.360738 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.360725    2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:38:22.426816 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.426758    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z6jld"
Jul 26 15:38:22.440429 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.440409    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de5f64f_1327_445c_bd32_2314e29b690a.slice/crio-0d10a974c513f52c825182d12875007d24ce8f6d6ef5f8ee04f89d8ca30f79b1 WatchSource:0}: Error finding container 0d10a974c513f52c825182d12875007d24ce8f6d6ef5f8ee04f89d8ca30f79b1: Status 404 returned error can't find the container with id 0d10a974c513f52c825182d12875007d24ce8f6d6ef5f8ee04f89d8ca30f79b1
Jul 26 15:38:22.443848 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.443834    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d7l7j"
Jul 26 15:38:22.450637 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.450616    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6g9f9"
Jul 26 15:38:22.451723 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.451704    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b441876693b98f22616c38ca505a9b.slice/crio-18039462be08865fc1104cc00b35baf274d29e5701230d610d2251345e6101dd WatchSource:0}: Error finding container 18039462be08865fc1104cc00b35baf274d29e5701230d610d2251345e6101dd: Status 404 returned error can't find the container with id 18039462be08865fc1104cc00b35baf274d29e5701230d610d2251345e6101dd
Jul 26 15:38:22.453749 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.453727    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3996da_64e2_474e_b2c7_1b255286357b.slice/crio-601b2c9ff37241e752dab988d3f5444f371f6841c77a32a50f3ce37eeca7437c WatchSource:0}: Error finding container 601b2c9ff37241e752dab988d3f5444f371f6841c77a32a50f3ce37eeca7437c: Status 404 returned error can't find the container with id 601b2c9ff37241e752dab988d3f5444f371f6841c77a32a50f3ce37eeca7437c
Jul 26 15:38:22.459555 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.459538    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33720d3_9a94_4ecf_9a95_e427352f11b1.slice/crio-2dfd32a15d4dffc5e9e44fdc12ba5319f9fe273a8879edee5a1672c586071394 WatchSource:0}: Error finding container 2dfd32a15d4dffc5e9e44fdc12ba5319f9fe273a8879edee5a1672c586071394: Status 404 returned error can't find the container with id 2dfd32a15d4dffc5e9e44fdc12ba5319f9fe273a8879edee5a1672c586071394
Jul 26 15:38:22.463526 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.462020    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n4z5c"
Jul 26 15:38:22.468751 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.468735    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb"
Jul 26 15:38:22.470525 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.470507    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b1d160_11ca_47b7_a1bc_1f36cceb9ce7.slice/crio-e73dcb3cf290278f60cd8e6bccadeb27bb72663d68db5ca67001cc31b9ae84b7 WatchSource:0}: Error finding container e73dcb3cf290278f60cd8e6bccadeb27bb72663d68db5ca67001cc31b9ae84b7: Status 404 returned error can't find the container with id e73dcb3cf290278f60cd8e6bccadeb27bb72663d68db5ca67001cc31b9ae84b7
Jul 26 15:38:22.476067 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.476046    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec31e446_d355_4bcd_b047_145753788e89.slice/crio-1abddf0eb4249f0031eb9052f57783a9354d9f4bc1dfbebd93e42de1c23ff88e WatchSource:0}: Error finding container 1abddf0eb4249f0031eb9052f57783a9354d9f4bc1dfbebd93e42de1c23ff88e: Status 404 returned error can't find the container with id 1abddf0eb4249f0031eb9052f57783a9354d9f4bc1dfbebd93e42de1c23ff88e
Jul 26 15:38:22.487363 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.487347    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9szlv"
Jul 26 15:38:22.494565 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.494548    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd998ee4d_b750_48d8_8fbd_2e086b48e1c5.slice/crio-adac413f400311b829ea8a676740e0c0c2e039521dfdfcd4f79dddc58adfa450 WatchSource:0}: Error finding container adac413f400311b829ea8a676740e0c0c2e039521dfdfcd4f79dddc58adfa450: Status 404 returned error can't find the container with id adac413f400311b829ea8a676740e0c0c2e039521dfdfcd4f79dddc58adfa450
Jul 26 15:38:22.505500 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.505481    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9fdhf"
Jul 26 15:38:22.513184 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.513161    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcd463c_c172_449b_9271_a3f2f30a133b.slice/crio-77eb963ec40a2b464f6d444ee3ea31a47ecc73982b8123c9e70b31269d58663b WatchSource:0}: Error finding container 77eb963ec40a2b464f6d444ee3ea31a47ecc73982b8123c9e70b31269d58663b: Status 404 returned error can't find the container with id 77eb963ec40a2b464f6d444ee3ea31a47ecc73982b8123c9e70b31269d58663b
Jul 26 15:38:22.533318 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.533297    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:22.537056 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.537043    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:22.542659 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.542638    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98395466_d07f_467b_b216_c46e5f2a9884.slice/crio-78993076dac4799edf77ee226f0fd7f39ae0a3d0608fbce774ae786e52438bde WatchSource:0}: Error finding container 78993076dac4799edf77ee226f0fd7f39ae0a3d0608fbce774ae786e52438bde: Status 404 returned error can't find the container with id 78993076dac4799edf77ee226f0fd7f39ae0a3d0608fbce774ae786e52438bde
Jul 26 15:38:22.545416 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:38:22.545394    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b953aa_b2b6_4ac4_a36b_94d60df89a5f.slice/crio-9f9ad106f74e335b0cdc79dbae66603894775f3fc5b5f9d31beeabe726324307 WatchSource:0}: Error finding container 9f9ad106f74e335b0cdc79dbae66603894775f3fc5b5f9d31beeabe726324307: Status 404 returned error can't find the container with id 9f9ad106f74e335b0cdc79dbae66603894775f3fc5b5f9d31beeabe726324307
Jul 26 15:38:22.546892 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.546875    2534 kuberuntime_manager.go:1459] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-t4kd8_kube-system(40b953aa-b2b6-4ac4-a36b-94d60df89a5f): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 26 15:38:22.546933 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.546904    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ErrImagePull: \"pull QPS exceeded\"" pod="kube-system/konnectivity-agent-t4kd8" podUID="40b953aa-b2b6-4ac4-a36b-94d60df89a5f"
Jul 26 15:38:22.716882 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.716799    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:22.717007 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.716937    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:22.717007 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.716992    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:23.716978625 +0000 UTC m=+2.901044740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:22.817384 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.817347    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:22.817576 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.817507    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:22.817576 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.817568    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:38:23.817549567 +0000 UTC m=+3.001615699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:22.918205 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:22.918171    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:22.918385 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.918359    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:22.918385 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.918381    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:22.918492 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.918394    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:22.918492 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:22.918455    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:38:23.918435607 +0000 UTC m=+3.102501725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:23.218810 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.218768    2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 15:33:22 +0000 UTC" deadline="2028-04-20 18:46:54.112866804 +0000 UTC"
Jul 26 15:38:23.218810 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.218810    2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15219h8m30.894061754s"
Jul 26 15:38:23.288407 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.288368    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:23.288558 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.288497    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:23.314930 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.313736    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t4kd8" event={"ID":"40b953aa-b2b6-4ac4-a36b-94d60df89a5f","Type":"ContainerStarted","Data":"9f9ad106f74e335b0cdc79dbae66603894775f3fc5b5f9d31beeabe726324307"}
Jul 26 15:38:23.324062 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.323806    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:39fb880a46d245719ae227a8b8e803ed4945a53bbfd7aef02be61d262527989a\\\": ErrImagePull: pull QPS exceeded\"" pod="kube-system/konnectivity-agent-t4kd8" podUID="40b953aa-b2b6-4ac4-a36b-94d60df89a5f"
Jul 26 15:38:23.326092 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.326003    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n4z5c" event={"ID":"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7","Type":"ContainerStarted","Data":"e73dcb3cf290278f60cd8e6bccadeb27bb72663d68db5ca67001cc31b9ae84b7"}
Jul 26 15:38:23.344396 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.344360    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerStarted","Data":"2dfd32a15d4dffc5e9e44fdc12ba5319f9fe273a8879edee5a1672c586071394"}
Jul 26 15:38:23.349093 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.349068    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d7l7j" event={"ID":"5b3996da-64e2-474e-b2c7-1b255286357b","Type":"ContainerStarted","Data":"601b2c9ff37241e752dab988d3f5444f371f6841c77a32a50f3ce37eeca7437c"}
Jul 26 15:38:23.363140 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.363107    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal" event={"ID":"c0b441876693b98f22616c38ca505a9b","Type":"ContainerStarted","Data":"18039462be08865fc1104cc00b35baf274d29e5701230d610d2251345e6101dd"}
Jul 26 15:38:23.374321 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.374295    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z6jld" event={"ID":"9de5f64f-1327-445c-bd32-2314e29b690a","Type":"ContainerStarted","Data":"0d10a974c513f52c825182d12875007d24ce8f6d6ef5f8ee04f89d8ca30f79b1"}
Jul 26 15:38:23.396587 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.396535    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal" event={"ID":"a5682356d1d816bbe296f83312dcd1ed","Type":"ContainerStarted","Data":"55ebc754fa57861da8250d71e6a482d6718faf3efc5badb2b2bd1bdb289e6af8"}
Jul 26 15:38:23.429144 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.429114    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"78993076dac4799edf77ee226f0fd7f39ae0a3d0608fbce774ae786e52438bde"}
Jul 26 15:38:23.444157 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.444125    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9fdhf" event={"ID":"fdcd463c-c172-449b-9271-a3f2f30a133b","Type":"ContainerStarted","Data":"77eb963ec40a2b464f6d444ee3ea31a47ecc73982b8123c9e70b31269d58663b"}
Jul 26 15:38:23.473735 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.473596    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9szlv" event={"ID":"d998ee4d-b750-48d8-8fbd-2e086b48e1c5","Type":"ContainerStarted","Data":"adac413f400311b829ea8a676740e0c0c2e039521dfdfcd4f79dddc58adfa450"}
Jul 26 15:38:23.489309 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.489203    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb" event={"ID":"ec31e446-d355-4bcd-b047-145753788e89","Type":"ContainerStarted","Data":"1abddf0eb4249f0031eb9052f57783a9354d9f4bc1dfbebd93e42de1c23ff88e"}
Jul 26 15:38:23.576187 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.576123    2534 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:23.725890 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.725808    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:23.726073 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.725991    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:23.726073 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.726068    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:25.726039322 +0000 UTC m=+4.910105456 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:23.826642 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.826607    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:23.826874 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.826804    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:23.826958 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.826875    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:38:25.826855528 +0000 UTC m=+5.010921690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:23.928123 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:23.927451    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:23.928123 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.927646    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:23.928123 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.927666    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:23.928123 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.927678    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:23.928123 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:23.927733    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:38:25.927716831 +0000 UTC m=+5.111782964 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:24.219738 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:24.219640    2534 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 15:33:22 +0000 UTC" deadline="2028-04-12 05:47:29.690290428 +0000 UTC"
Jul 26 15:38:24.219738 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:24.219697    2534 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15014h9m5.470597168s"
Jul 26 15:38:24.288665 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:24.288635    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:24.288834 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:24.288774    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:24.289193 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:24.289173    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:24.289310 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:24.289290    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:25.049889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:25.049660    2534 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 15:38:25.290293 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:25.290084    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:25.290293 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.290203    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:25.742374 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:25.741891    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:25.742374 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.742086    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:25.742374 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.742153    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:29.742135194 +0000 UTC m=+8.926201322 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:25.843081 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:25.843000    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:25.843268 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.843174    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:25.843268 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.843257    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:38:29.843223746 +0000 UTC m=+9.027289861 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:25.943907 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:25.943867    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:25.944105 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.944052    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:25.944105 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.944078    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:25.944105 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.944091    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:25.944305 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:25.944150    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:38:29.944130826 +0000 UTC m=+9.128196946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:26.288737 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:26.288660    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:26.288899 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:26.288798    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:26.289212 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:26.289183    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:26.289364 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:26.289321    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:27.291506 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:27.291476    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:27.291954 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:27.291597    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:28.288693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:28.288018    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:28.288693 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:28.288148    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:28.288693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:28.288552    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:28.288693 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:28.288652    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:29.290948 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:29.290913    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:29.291406 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.291040    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:29.778413 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:29.778375    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:29.778601 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.778529    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:29.778601 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.778586    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:37.778567298 +0000 UTC m=+16.962633418 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:29.879728 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:29.879690    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:29.879914 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.879882    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:29.879976 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.879941    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:38:37.879923947 +0000 UTC m=+17.063990065 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:29.980940 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:29.980904    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:29.981116 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.981079    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:29.981116 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.981100    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:29.981116 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.981113    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:29.981303 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:29.981174    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:38:37.981156414 +0000 UTC m=+17.165222534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:30.288002 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:30.287971    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:30.288159 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:30.288097    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:30.288250 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:30.288162    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:30.288300 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:30.288257    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:31.292284 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:31.291754    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:31.292284 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:31.291856    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:32.288110 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:32.288073    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:32.288322 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:32.288119    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:32.288322 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:32.288211    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:32.288443 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:32.288328    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:33.290994 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:33.290956    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:33.291451 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:33.291079    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:34.288110 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:34.288078    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:34.288307 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:34.288123    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:34.288307 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:34.288205    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:34.288396 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:34.288345    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:35.291340 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:35.291306    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:35.291814 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:35.291436    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:36.288410 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:36.288378    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:36.288589 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:36.288467    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:36.288664 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:36.288584    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:36.288731 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:36.288708    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:37.291135 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:37.291106    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:37.291541 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:37.291251    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:37.835345 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:37.835310    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:37.835528 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:37.835470    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:37.835591 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:37.835531    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:38:53.835516927 +0000 UTC m=+33.019583046 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:37.935619 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:37.935586    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:37.935781 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:37.935744    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:37.935829 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:37.935814    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:38:53.935794901 +0000 UTC m=+33.119861039 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:38.036290 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:38.036251    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:38.036467 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:38.036359    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:38.036467 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:38.036381    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:38.036467 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:38.036391    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:38.036467 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:38.036447    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:38:54.036426375 +0000 UTC m=+33.220492503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:38.288009 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:38.287967    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:38.288190 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:38.288015    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:38.288190 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:38.288122    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:38.288322 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:38.288262    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:39.287971 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:39.287942    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:39.288473 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:39.288069    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:40.287912 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:40.287880    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:40.288105 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:40.288040    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:40.288105 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:40.288086    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:40.288502 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:40.288148    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:41.289029 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.288700    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:41.289029 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:41.288975    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:41.531199 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.531008    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z6jld" event={"ID":"9de5f64f-1327-445c-bd32-2314e29b690a","Type":"ContainerStarted","Data":"0688dac6c285aa1e2bd44e2d145c334247ed5267c70277475958bd20e1d46045"}
Jul 26 15:38:41.535917 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.535863    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal" event={"ID":"a5682356d1d816bbe296f83312dcd1ed","Type":"ContainerStarted","Data":"b40d0bd2ba60635ca956bcc7634ac1745f3a4115f0327bd7af0dc2d5486052c4"}
Jul 26 15:38:41.546882 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.546815    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z6jld" podStartSLOduration=1.8096915980000001 podStartE2EDuration="20.546796779s" totalImagesPullingTime="18.737105181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:41.546629584 +0000 UTC m=+20.730695723" watchObservedRunningTime="2026-07-26 15:38:41.546796779 +0000 UTC m=+20.730862936"
Jul 26 15:38:41.547342 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.547282    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"3b4f9f3a7af68136af8bcc4e54df14e01edf45476f6a131d59044193433d2d8c"}
Jul 26 15:38:41.547459 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.547372    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"a2e4004ffdf327f9348021b4bea44a7e8fd10027a74b8abc145882043cb586d1"}
Jul 26 15:38:41.555488 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.555372    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n4z5c" event={"ID":"93b1d160-11ca-47b7-a1bc-1f36cceb9ce7","Type":"ContainerStarted","Data":"ca6046cb8a3e9ef402bc289258b20ac12ca80432dba7540ad3d2d7dd53a386ee"}
Jul 26 15:38:41.577809 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.577749    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-n4z5c" podStartSLOduration=1.741476351 podStartE2EDuration="20.577712894s" totalImagesPullingTime="18.836236543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:41.577405682 +0000 UTC m=+20.761471820" watchObservedRunningTime="2026-07-26 15:38:41.577712894 +0000 UTC m=+20.761779038"
Jul 26 15:38:41.578014 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:41.577977    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-109.ec2.internal" podStartSLOduration=19.577967077 podStartE2EDuration="19.577967077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:41.562112056 +0000 UTC m=+20.746178194" watchObservedRunningTime="2026-07-26 15:38:41.577967077 +0000 UTC m=+20.762033215"
Jul 26 15:38:42.287910 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.287881    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:42.288084 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.287887    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:42.288084 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:42.288001    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:42.288199 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:42.288104    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:42.559167 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.559090    2534 generic.go:356] "Generic (PLEG): container finished" podID="d33720d3-9a94-4ecf-9a95-e427352f11b1" containerID="6d6c1c9bbeffd2a4879ec6bf2b8a07d58547dc9fa0726e525d3af01f7b1c0934" exitCode=0
Jul 26 15:38:42.559853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.559168    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerDied","Data":"6d6c1c9bbeffd2a4879ec6bf2b8a07d58547dc9fa0726e525d3af01f7b1c0934"}
Jul 26 15:38:42.560873 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.560826    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d7l7j" event={"ID":"5b3996da-64e2-474e-b2c7-1b255286357b","Type":"ContainerStarted","Data":"5c1901356ac8fb081389f0bdcae33c0b9aedd2e64121ff31d5aca8f5ee837d50"}
Jul 26 15:38:42.562537 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.562513    2534 generic.go:356] "Generic (PLEG): container finished" podID="c0b441876693b98f22616c38ca505a9b" containerID="8b51cd595de37283681f8fbd7e81b5e42b5f6514573487aea8a49dbcc3c95404" exitCode=0
Jul 26 15:38:42.562641 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.562596    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal" event={"ID":"c0b441876693b98f22616c38ca505a9b","Type":"ContainerDied","Data":"8b51cd595de37283681f8fbd7e81b5e42b5f6514573487aea8a49dbcc3c95404"}
Jul 26 15:38:42.565999 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.565973    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"fbf8db59e7be0ce137391d34443def82e72f3e758a171edd0b7d2a4d1369d33a"}
Jul 26 15:38:42.566096 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.566057    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"4bf7e691235461bf22b532ceb6275a68502e2eb4f9417ccaa58aef1a130bf762"}
Jul 26 15:38:42.566151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.566108    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"fe86838e65626c5ae3160a2a8917b79aa54536b0ceacd532b0573282951f45ab"}
Jul 26 15:38:42.566206 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.566157    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"775ee60030be63dc5c4ca895936fecb9b8128fd2b5b6439968d999f918dd0a11"}
Jul 26 15:38:42.567410 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.567379    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9szlv" event={"ID":"d998ee4d-b750-48d8-8fbd-2e086b48e1c5","Type":"ContainerStarted","Data":"2e06f767e49b8d190135131be8132ca6ded795f5309da56b777190ee29e5728f"}
Jul 26 15:38:42.568718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.568697    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb" event={"ID":"ec31e446-d355-4bcd-b047-145753788e89","Type":"ContainerStarted","Data":"eae054ac9b313e60eeb93c36ea26292bec0c0e3b00aac8f15badf9eab4d47a6d"}
Jul 26 15:38:42.604285 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.604226    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d7l7j" podStartSLOduration=2.881076611 podStartE2EDuration="21.604212297s" totalImagesPullingTime="18.723135686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:42.603780794 +0000 UTC m=+21.787846922" watchObservedRunningTime="2026-07-26 15:38:42.604212297 +0000 UTC m=+21.788278433"
Jul 26 15:38:42.615896 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:42.615853    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9szlv" podStartSLOduration=7.26517596 podStartE2EDuration="21.615839483s" totalImagesPullingTime="14.350663523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:42.61556531 +0000 UTC m=+21.799631448" watchObservedRunningTime="2026-07-26 15:38:42.615839483 +0000 UTC m=+21.799905620"
Jul 26 15:38:43.288367 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.288104    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:43.288507 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:43.288479    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:43.356547 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.356520    2534 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 15:38:43.574668 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.574592    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal" event={"ID":"c0b441876693b98f22616c38ca505a9b","Type":"ContainerStarted","Data":"502d4dfdc51e302867946fe1238e356079b91e0ddd0f954ec6eee94996b0f552"}
Jul 26 15:38:43.576375 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.576347    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9fdhf" event={"ID":"fdcd463c-c172-449b-9271-a3f2f30a133b","Type":"ContainerStarted","Data":"6444ba640e332644cef7738f5b3ed4e6274808cac6bff6af9a1914f26dd60337"}
Jul 26 15:38:43.578590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.578565    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb" event={"ID":"ec31e446-d355-4bcd-b047-145753788e89","Type":"ContainerStarted","Data":"16cbc3fde9e2ab5aa7b657dbd7601ae16f8d46af7bee3839aba594186859d859"}
Jul 26 15:38:43.580012 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.579983    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-t4kd8" event={"ID":"40b953aa-b2b6-4ac4-a36b-94d60df89a5f","Type":"ContainerStarted","Data":"ea392ca236b9a116b07c77d318d3b1098224c81fef6671405cad597a4efdbca3"}
Jul 26 15:38:43.591387 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.591339    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-109.ec2.internal" podStartSLOduration=21.591325843 podStartE2EDuration="21.591325843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:43.590766581 +0000 UTC m=+22.774832719" watchObservedRunningTime="2026-07-26 15:38:43.591325843 +0000 UTC m=+22.775391978"
Jul 26 15:38:43.603497 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.603460    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9fdhf" podStartSLOduration=3.938445701 podStartE2EDuration="22.603443434s" totalImagesPullingTime="18.664997733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:43.60307321 +0000 UTC m=+22.787139548" watchObservedRunningTime="2026-07-26 15:38:43.603443434 +0000 UTC m=+22.787509572"
Jul 26 15:38:43.615152 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:43.615110    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-t4kd8" podStartSLOduration=2.252895112 podStartE2EDuration="22.615101839s" totalImagesPullingTime="20.362206727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-26 15:38:43.61505322 +0000 UTC m=+22.799119362" watchObservedRunningTime="2026-07-26 15:38:43.615101839 +0000 UTC m=+22.799167975"
Jul 26 15:38:44.175320 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:44.175179    2534 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T15:38:43.356543735Z","UUID":"c2c50322-ad06-4fc4-84e2-188a863b02ed","Handler":null,"Name":"","Endpoint":""}
Jul 26 15:38:44.176917 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:44.176812    2534 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 15:38:44.176917 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:44.176856    2534 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 15:38:44.288664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:44.288631    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:44.288664 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:44.288655    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:44.288884 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:44.288786    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:44.288956 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:44.288930    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:44.499053 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:44.499028    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9szlv_d998ee4d-b750-48d8-8fbd-2e086b48e1c5/dns-node-resolver/0.log"
Jul 26 15:38:45.288716 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:45.288676    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:45.289125 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:45.288801    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:45.588683 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:45.588599    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"302d2ee53f7a6ba283c1619ba39558347fd1957210ee75de7b1d1cadf28dad55"}
Jul 26 15:38:45.590782 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:45.590741    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb" event={"ID":"ec31e446-d355-4bcd-b047-145753788e89","Type":"ContainerStarted","Data":"a3784e05c8039f614f6961d0835f4301fcabd6bf1dc78762f682daa01fded610"}
Jul 26 15:38:45.608907 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:45.608856    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9mkfb" podStartSLOduration=3.14421991 podStartE2EDuration="24.608839278s" totalImagesPullingTime="21.464619368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:45.608688541 +0000 UTC m=+24.792754680" watchObservedRunningTime="2026-07-26 15:38:45.608839278 +0000 UTC m=+24.792905415"
Jul 26 15:38:45.674545 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:45.674514    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d7l7j_5b3996da-64e2-474e-b2c7-1b255286357b/node-ca/0.log"
Jul 26 15:38:46.081219 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:46.081179    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:46.081863 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:46.081841    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:46.288393 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:46.288362    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:46.288393 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:46.288402    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:46.288630 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:46.288513    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:46.288697 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:46.288667    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:46.593065 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:46.593029    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:46.593511 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:46.593446    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-t4kd8"
Jul 26 15:38:47.290842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.290678    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:47.291008 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:47.290921    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:47.596300 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.596268    2534 generic.go:356] "Generic (PLEG): container finished" podID="d33720d3-9a94-4ecf-9a95-e427352f11b1" containerID="2f048c33ea2f9ec9476bc81050d86086141aa4b3480964576ad213981494942f" exitCode=0
Jul 26 15:38:47.596951 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.596353    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerDied","Data":"2f048c33ea2f9ec9476bc81050d86086141aa4b3480964576ad213981494942f"}
Jul 26 15:38:47.600278 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.600255    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" event={"ID":"98395466-d07f-467b-b216-c46e5f2a9884","Type":"ContainerStarted","Data":"b385b2f38e7aedf28cbb77d6a80fc9a9898e8488bf24fbd1b9fc1717189b6a55"}
Jul 26 15:38:47.600588 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.600565    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:47.600702 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.600637    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:47.600702 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.600687    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:47.615580 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.615560    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:47.615697 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.615687    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:38:47.639690 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:47.639651    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2" podStartSLOduration=7.963441645 podStartE2EDuration="26.639639574s" totalImagesPullingTime="18.676197929s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:47.638881932 +0000 UTC m=+26.822948069" watchObservedRunningTime="2026-07-26 15:38:47.639639574 +0000 UTC m=+26.823705712"
Jul 26 15:38:48.288364 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:48.288332    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:48.288364 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:48.288352    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:48.288608 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:48.288442    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:48.288608 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:48.288558    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:49.287761 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:49.287728    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:49.288305 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:49.287824    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:49.606402 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:49.606369    2534 generic.go:356] "Generic (PLEG): container finished" podID="d33720d3-9a94-4ecf-9a95-e427352f11b1" containerID="f03ae0df49dc36a1aee4945f45b7f0da3df0f8a1f88265fc47da7e157f6e4927" exitCode=0
Jul 26 15:38:49.606562 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:49.606448    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerDied","Data":"f03ae0df49dc36a1aee4945f45b7f0da3df0f8a1f88265fc47da7e157f6e4927"}
Jul 26 15:38:50.288017 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:50.287985    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:50.288455 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:50.288024    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:50.288455 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:50.288133    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:50.288455 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:50.288255    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:51.291080 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:51.290924    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:51.291449 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:51.291154    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:51.612814 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:51.612729    2534 generic.go:356] "Generic (PLEG): container finished" podID="d33720d3-9a94-4ecf-9a95-e427352f11b1" containerID="d9a5d98b4533e724eddd1690eb2ae06a7ff5a6c80279c8869e7564d992df56db" exitCode=0
Jul 26 15:38:51.612814 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:51.612804    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerDied","Data":"d9a5d98b4533e724eddd1690eb2ae06a7ff5a6c80279c8869e7564d992df56db"}
Jul 26 15:38:52.287635 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:52.287609    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:52.287787 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:52.287707    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:52.287787 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:52.287760    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:52.287878 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:52.287859    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:53.288243 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:53.288211    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:53.288675 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:53.288354    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:53.866120 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:53.866085    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:53.866289 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:53.866197    2534 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:53.866289 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:53.866264    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs podName:0590d803-5bd9-4739-abf3-13c8bdacc2c0 nodeName:}" failed. No retries permitted until 2026-07-26 15:39:25.866247943 +0000 UTC m=+65.050314073 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs") pod "network-metrics-daemon-tfwln" (UID: "0590d803-5bd9-4739-abf3-13c8bdacc2c0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 15:38:53.966722 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:53.966691    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:53.966874 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:53.966837    2534 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:53.966947 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:53.966908    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret podName:51eb5653-f3da-4900-8f1d-790a4ec88c7d nodeName:}" failed. No retries permitted until 2026-07-26 15:39:25.966886918 +0000 UTC m=+65.150953057 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret") pod "global-pull-secret-syncer-f5rzd" (UID: "51eb5653-f3da-4900-8f1d-790a4ec88c7d") : object "kube-system"/"original-pull-secret" not registered
Jul 26 15:38:54.067053 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:54.067022    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:54.067200 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:54.067149    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 15:38:54.067200 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:54.067166    2534 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 15:38:54.067200 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:54.067176    2534 projected.go:196] Error preparing data for projected volume kube-api-access-7vt67 for pod openshift-network-diagnostics/network-check-target-zkvdb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:54.067317 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:54.067226    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67 podName:9f7ff9d6-945b-462b-b3d4-425473eece9e nodeName:}" failed. No retries permitted until 2026-07-26 15:39:26.067210907 +0000 UTC m=+65.251277025 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7vt67" (UniqueName: "kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67") pod "network-check-target-zkvdb" (UID: "9f7ff9d6-945b-462b-b3d4-425473eece9e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 15:38:54.287801 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:54.287766    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:54.287968 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:54.287776    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:54.287968 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:54.287871    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:54.288048 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:54.287974    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:55.288277 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:55.288245    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:55.288733 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:55.288375    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:56.288376 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:56.288346    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:56.288719 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:56.288346    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:56.288719 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:56.288448    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:56.288719 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:56.288524    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:56.626654 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:56.626563    2534 generic.go:356] "Generic (PLEG): container finished" podID="d33720d3-9a94-4ecf-9a95-e427352f11b1" containerID="71d6c3478d0882df6e06a5d8727b44464bfbe060cf1be5e5dc4e9ddf0cabb876" exitCode=0
Jul 26 15:38:56.626654 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:56.626602    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerDied","Data":"71d6c3478d0882df6e06a5d8727b44464bfbe060cf1be5e5dc4e9ddf0cabb876"}
Jul 26 15:38:57.288318 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:57.288292    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:57.288512 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:57.288388    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:38:57.631885 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:57.631801    2534 generic.go:356] "Generic (PLEG): container finished" podID="d33720d3-9a94-4ecf-9a95-e427352f11b1" containerID="e3e0526ff160ee1716acecc7b59633bd29b547fe137989b6fefa379bf4843f51" exitCode=0
Jul 26 15:38:57.631885 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:57.631856    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerDied","Data":"e3e0526ff160ee1716acecc7b59633bd29b547fe137989b6fefa379bf4843f51"}
Jul 26 15:38:58.287672 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:58.287644    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:38:58.287843 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:58.287735    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:38:58.287843 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:58.287792    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:38:58.287924 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:58.287881    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:38:58.637050 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:58.636977    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" event={"ID":"d33720d3-9a94-4ecf-9a95-e427352f11b1","Type":"ContainerStarted","Data":"2e0d353ce8ad27c8875a8d5579305b034623e178b922b1ecb3e046b8f9bf5873"}
Jul 26 15:38:58.657013 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:58.656961    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6g9f9" podStartSLOduration=8.354345382 podStartE2EDuration="37.656947579s" totalImagesPullingTime="29.302602197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:38:58.655953891 +0000 UTC m=+37.840020025" watchObservedRunningTime="2026-07-26 15:38:58.656947579 +0000 UTC m=+37.841013715"
Jul 26 15:38:59.288183 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:38:59.288149    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:38:59.288372 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:38:59.288279    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:00.287685 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:00.287657    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:00.288048 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:00.287769    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:00.288048 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:00.287815    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:00.288048 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:00.287914    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:01.289299 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:01.289270    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:01.289665 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:01.289348    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:02.288524 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:02.288489    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:02.288699 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:02.288591    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:02.288699 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:02.288647    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:02.288817 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:02.288757    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:03.288050 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:03.288017    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:03.288443 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:03.288121    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:04.288432 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:04.288396    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:04.288950 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:04.288408    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:04.288950 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:04.288491    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:04.288950 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:04.288574    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:05.288675 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:05.288644    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:05.289036 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:05.288743    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:06.287994 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:06.287963    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:06.288207 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:06.287966    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:06.288207 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:06.288160    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:06.288207 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:06.288055    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:07.288193 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:07.288161    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:07.288620 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:07.288279    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:08.287735 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:08.287706    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:08.287735 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:08.287736    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:08.287972 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:08.287805    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:08.287972 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:08.287891    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:09.290655 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:09.290625    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:09.291050 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:09.290723    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:10.287638 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.287605    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:10.287806 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.287722    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:10.287868 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:10.287815    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:10.287868 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:10.287717    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:10.826649 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.826610    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f5rzd"]
Jul 26 15:39:10.827100 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.826715    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:10.827100 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:10.826805    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:10.829938 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.829890    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkvdb"]
Jul 26 15:39:10.830196 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.830177    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:10.830345 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:10.830313    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:10.830740 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.830708    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tfwln"]
Jul 26 15:39:10.830819 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:10.830791    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:10.830883 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:10.830865    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:12.287956 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:12.287791    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:12.288412 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:12.287791    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:12.288412 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:12.288030    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:12.288412 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:12.288138    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:13.288115 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:13.288088    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:13.288575 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:13.288192    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f5rzd" podUID="51eb5653-f3da-4900-8f1d-790a4ec88c7d"
Jul 26 15:39:14.287748 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:14.287718    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:14.287927 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:14.287724    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:14.287927 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:14.287827    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tfwln" podUID="0590d803-5bd9-4739-abf3-13c8bdacc2c0"
Jul 26 15:39:14.287927 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:14.287902    2534 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zkvdb" podUID="9f7ff9d6-945b-462b-b3d4-425473eece9e"
Jul 26 15:39:15.156679 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.156612    2534 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-109.ec2.internal" event="NodeReady"
Jul 26 15:39:15.157020 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.156718    2534 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 15:39:15.205298 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.205267    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6tbt2"]
Jul 26 15:39:15.208027 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.208011    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.210799 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.210765    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 15:39:15.211156 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.211132    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t6bwt\""
Jul 26 15:39:15.211332 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.211163    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 15:39:15.211556 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.211538    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8xmfz"]
Jul 26 15:39:15.214176 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.214159    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.216295 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.216274    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fwcq5\""
Jul 26 15:39:15.216295 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.216285    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 15:39:15.216444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.216303    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 15:39:15.216444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.216359    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 15:39:15.216444 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.216408    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 15:39:15.219537 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.219512    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6tbt2"]
Jul 26 15:39:15.225823 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.225434    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8xmfz"]
Jul 26 15:39:15.232491 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232469    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/514e1705-96ac-435a-b371-d2fbcfa26998-crio-socket\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.232599 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232508    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/514e1705-96ac-435a-b371-d2fbcfa26998-data-volume\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.232599 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232538    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514e1705-96ac-435a-b371-d2fbcfa26998-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.232599 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232563    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514e1705-96ac-435a-b371-d2fbcfa26998-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.232599 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232593    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-tmp-dir\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.232768 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232616    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pnck\" (UniqueName: \"kubernetes.io/projected/514e1705-96ac-435a-b371-d2fbcfa26998-kube-api-access-6pnck\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.232768 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232678    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-metrics-tls\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.232768 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232720    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfdm8\" (UniqueName: \"kubernetes.io/projected/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-kube-api-access-sfdm8\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.232857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.232808    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-config-volume\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.288008 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.287977    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:15.290351 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.290332    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 15:39:15.310369 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.310346    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kv5pj"]
Jul 26 15:39:15.313137 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.313123    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.315370 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.315349    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-b8h2r\""
Jul 26 15:39:15.315489 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.315464    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 15:39:15.315591 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.315512    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 15:39:15.315654 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.315604    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 15:39:15.321079 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.321060    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kv5pj"]
Jul 26 15:39:15.333854 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.333835    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-tmp-dir\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.333951 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.333861    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pnck\" (UniqueName: \"kubernetes.io/projected/514e1705-96ac-435a-b371-d2fbcfa26998-kube-api-access-6pnck\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.333951 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.333882    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-metrics-tls\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.333951 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.333899    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfdm8\" (UniqueName: \"kubernetes.io/projected/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-kube-api-access-sfdm8\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.333951 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.333947    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-config-volume\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.334151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334013    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/514e1705-96ac-435a-b371-d2fbcfa26998-crio-socket\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.334151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334037    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb0601-03c3-4da0-ac2b-0baa98477acc-cert\") pod \"ingress-canary-kv5pj\" (UID: \"d6fb0601-03c3-4da0-ac2b-0baa98477acc\") " pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.334151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334060    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9rp5\" (UniqueName: \"kubernetes.io/projected/d6fb0601-03c3-4da0-ac2b-0baa98477acc-kube-api-access-z9rp5\") pod \"ingress-canary-kv5pj\" (UID: \"d6fb0601-03c3-4da0-ac2b-0baa98477acc\") " pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.334151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334096    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/514e1705-96ac-435a-b371-d2fbcfa26998-data-volume\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.334151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334117    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/514e1705-96ac-435a-b371-d2fbcfa26998-crio-socket\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.334151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334129    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514e1705-96ac-435a-b371-d2fbcfa26998-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.334420 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334163    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514e1705-96ac-435a-b371-d2fbcfa26998-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.334420 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334333    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-tmp-dir\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.334511 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334493    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/514e1705-96ac-435a-b371-d2fbcfa26998-data-volume\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.334660 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334634    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-config-volume\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.334960 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.334906    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/514e1705-96ac-435a-b371-d2fbcfa26998-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.337920 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.337903    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-metrics-tls\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.337920 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.337917    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/514e1705-96ac-435a-b371-d2fbcfa26998-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.341285 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.341260    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pnck\" (UniqueName: \"kubernetes.io/projected/514e1705-96ac-435a-b371-d2fbcfa26998-kube-api-access-6pnck\") pod \"insights-runtime-extractor-8xmfz\" (UID: \"514e1705-96ac-435a-b371-d2fbcfa26998\") " pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.341809 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.341786    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfdm8\" (UniqueName: \"kubernetes.io/projected/fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65-kube-api-access-sfdm8\") pod \"dns-default-6tbt2\" (UID: \"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65\") " pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.434541 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.434476    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb0601-03c3-4da0-ac2b-0baa98477acc-cert\") pod \"ingress-canary-kv5pj\" (UID: \"d6fb0601-03c3-4da0-ac2b-0baa98477acc\") " pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.434541 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.434505    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9rp5\" (UniqueName: \"kubernetes.io/projected/d6fb0601-03c3-4da0-ac2b-0baa98477acc-kube-api-access-z9rp5\") pod \"ingress-canary-kv5pj\" (UID: \"d6fb0601-03c3-4da0-ac2b-0baa98477acc\") " pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.436615 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.436594    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb0601-03c3-4da0-ac2b-0baa98477acc-cert\") pod \"ingress-canary-kv5pj\" (UID: \"d6fb0601-03c3-4da0-ac2b-0baa98477acc\") " pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.441845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.441825    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9rp5\" (UniqueName: \"kubernetes.io/projected/d6fb0601-03c3-4da0-ac2b-0baa98477acc-kube-api-access-z9rp5\") pod \"ingress-canary-kv5pj\" (UID: \"d6fb0601-03c3-4da0-ac2b-0baa98477acc\") " pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.517753 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.517722    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:15.524827 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.524812    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8xmfz"
Jul 26 15:39:15.621540 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.621513    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kv5pj"
Jul 26 15:39:15.656473 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.656445    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6tbt2"]
Jul 26 15:39:15.659760 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:15.659723    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd5a0fe_9e63_4f61_9b20_a5c4fa92ea65.slice/crio-6e390879c45f689315a249c5452f7ebe3f20128b2b314eb79ea4ec488657a2b5 WatchSource:0}: Error finding container 6e390879c45f689315a249c5452f7ebe3f20128b2b314eb79ea4ec488657a2b5: Status 404 returned error can't find the container with id 6e390879c45f689315a249c5452f7ebe3f20128b2b314eb79ea4ec488657a2b5
Jul 26 15:39:15.670849 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.670823    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8xmfz"]
Jul 26 15:39:15.673924 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.673750    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6tbt2" event={"ID":"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65","Type":"ContainerStarted","Data":"6e390879c45f689315a249c5452f7ebe3f20128b2b314eb79ea4ec488657a2b5"}
Jul 26 15:39:15.676484 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:15.676458    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514e1705_96ac_435a_b371_d2fbcfa26998.slice/crio-cf84d6151dd6ea9e68f1137dafda76d2ec26a1eca5e1568f1305a3b56565b93c WatchSource:0}: Error finding container cf84d6151dd6ea9e68f1137dafda76d2ec26a1eca5e1568f1305a3b56565b93c: Status 404 returned error can't find the container with id cf84d6151dd6ea9e68f1137dafda76d2ec26a1eca5e1568f1305a3b56565b93c
Jul 26 15:39:15.746195 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:15.746164    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kv5pj"]
Jul 26 15:39:15.749216 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:15.749192    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fb0601_03c3_4da0_ac2b_0baa98477acc.slice/crio-d6f36d2bbd095d2d4e3fb1ee8472f9ba584b74288cdbf1ddcc98f4e2bbb074ed WatchSource:0}: Error finding container d6f36d2bbd095d2d4e3fb1ee8472f9ba584b74288cdbf1ddcc98f4e2bbb074ed: Status 404 returned error can't find the container with id d6f36d2bbd095d2d4e3fb1ee8472f9ba584b74288cdbf1ddcc98f4e2bbb074ed
Jul 26 15:39:16.289556 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.288148    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:16.289556 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.288707    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:16.296012 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.294122    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 15:39:16.296012 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.294420    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pkvmj\""
Jul 26 15:39:16.296012 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.294637    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 15:39:16.296801 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.296445    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8j62j\""
Jul 26 15:39:16.296801 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.296642    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 15:39:16.678996 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.678878    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kv5pj" event={"ID":"d6fb0601-03c3-4da0-ac2b-0baa98477acc","Type":"ContainerStarted","Data":"d6f36d2bbd095d2d4e3fb1ee8472f9ba584b74288cdbf1ddcc98f4e2bbb074ed"}
Jul 26 15:39:16.680495 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:16.680421    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmfz" event={"ID":"514e1705-96ac-435a-b371-d2fbcfa26998","Type":"ContainerStarted","Data":"cf84d6151dd6ea9e68f1137dafda76d2ec26a1eca5e1568f1305a3b56565b93c"}
Jul 26 15:39:19.618365 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:19.618336    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gbw2"
Jul 26 15:39:22.698536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.698487    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6tbt2" event={"ID":"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65","Type":"ContainerStarted","Data":"fc126f0408ba0b6ec63f9de664fcfe69920e4c5bcdbab8d7dc47239f8edecd4b"}
Jul 26 15:39:22.698944 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.698590    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6tbt2" event={"ID":"fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65","Type":"ContainerStarted","Data":"62bf6375477857cfcd8dec4195df519febbf1d8ee2504d15e3bb14255ced6f12"}
Jul 26 15:39:22.699555 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.699366    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:22.701357 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.701306    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kv5pj" event={"ID":"d6fb0601-03c3-4da0-ac2b-0baa98477acc","Type":"ContainerStarted","Data":"03a8380b717df5150d25bef2a379137b6ac8cdc60c144ed7958396a486d0c74f"}
Jul 26 15:39:22.703330 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.703308    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmfz" event={"ID":"514e1705-96ac-435a-b371-d2fbcfa26998","Type":"ContainerStarted","Data":"fc8599531df8002b693e6438b8baa47f9e10b8b5b64f861497ed34b05b3fa1ff"}
Jul 26 15:39:22.715782 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.715734    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6tbt2" podStartSLOduration=1.65642388 podStartE2EDuration="7.715717502s" totalImagesPullingTime="6.059293622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:22.714593208 +0000 UTC m=+61.898659346" watchObservedRunningTime="2026-07-26 15:39:22.715717502 +0000 UTC m=+61.899783638"
Jul 26 15:39:22.731847 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:22.731800    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kv5pj" podStartSLOduration=1.762114232 podStartE2EDuration="7.731787838s" totalImagesPullingTime="5.969673606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:22.731023392 +0000 UTC m=+61.915089530" watchObservedRunningTime="2026-07-26 15:39:22.731787838 +0000 UTC m=+61.915853953"
Jul 26 15:39:24.373391 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.373356    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"]
Jul 26 15:39:24.398116 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.398089    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"]
Jul 26 15:39:24.398262 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.398210    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.400923 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.400900    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 15:39:24.400923 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.400905    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-4lvfd\""
Jul 26 15:39:24.401109 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.401013    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 26 15:39:24.401157 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.401112    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 26 15:39:24.401752 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.401706    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 15:39:24.401890 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.401717    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 15:39:24.503251 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.503202    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1650e062-7149-4008-9c55-c512f46a9606-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.503417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.503352    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1650e062-7149-4008-9c55-c512f46a9606-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.503417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.503381    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1650e062-7149-4008-9c55-c512f46a9606-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.503417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.503406    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2rrg\" (UniqueName: \"kubernetes.io/projected/1650e062-7149-4008-9c55-c512f46a9606-kube-api-access-r2rrg\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.603927 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.603890    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1650e062-7149-4008-9c55-c512f46a9606-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.604102 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.603970    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1650e062-7149-4008-9c55-c512f46a9606-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.604102 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.603997    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1650e062-7149-4008-9c55-c512f46a9606-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.604102 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.604024    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2rrg\" (UniqueName: \"kubernetes.io/projected/1650e062-7149-4008-9c55-c512f46a9606-kube-api-access-r2rrg\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.604820 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.604795    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1650e062-7149-4008-9c55-c512f46a9606-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.606683 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.606654    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1650e062-7149-4008-9c55-c512f46a9606-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.606786 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.606711    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1650e062-7149-4008-9c55-c512f46a9606-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.612127 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.612081    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2rrg\" (UniqueName: \"kubernetes.io/projected/1650e062-7149-4008-9c55-c512f46a9606-kube-api-access-r2rrg\") pod \"prometheus-operator-84d69f794d-qzcl2\" (UID: \"1650e062-7149-4008-9c55-c512f46a9606\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:24.706858 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:24.706823    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"
Jul 26 15:39:25.218436 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:25.218388    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-qzcl2"]
Jul 26 15:39:25.222380 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:25.222352    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1650e062_7149_4008_9c55_c512f46a9606.slice/crio-d64ceb678d81ddc4132e948dfbbe4cd73fa3ff3aaaea2e0c51259521b952cd55 WatchSource:0}: Error finding container d64ceb678d81ddc4132e948dfbbe4cd73fa3ff3aaaea2e0c51259521b952cd55: Status 404 returned error can't find the container with id d64ceb678d81ddc4132e948dfbbe4cd73fa3ff3aaaea2e0c51259521b952cd55
Jul 26 15:39:25.714826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:25.714790    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmfz" event={"ID":"514e1705-96ac-435a-b371-d2fbcfa26998","Type":"ContainerStarted","Data":"210383e8e7090ee783fccae4a0ea7d87ace2340c77e673aceabb83c0b7dcb1f8"}
Jul 26 15:39:25.716497 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:25.716463    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2" event={"ID":"1650e062-7149-4008-9c55-c512f46a9606","Type":"ContainerStarted","Data":"d64ceb678d81ddc4132e948dfbbe4cd73fa3ff3aaaea2e0c51259521b952cd55"}
Jul 26 15:39:25.913715 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:25.913679    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:25.916011 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:25.915970    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 15:39:25.927278 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:25.927255    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0590d803-5bd9-4739-abf3-13c8bdacc2c0-metrics-certs\") pod \"network-metrics-daemon-tfwln\" (UID: \"0590d803-5bd9-4739-abf3-13c8bdacc2c0\") " pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:26.014700 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.014612    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:26.017262 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.017225    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 15:39:26.027866 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.027824    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/51eb5653-f3da-4900-8f1d-790a4ec88c7d-original-pull-secret\") pod \"global-pull-secret-syncer-f5rzd\" (UID: \"51eb5653-f3da-4900-8f1d-790a4ec88c7d\") " pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:26.096205 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.096170    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f5rzd"
Jul 26 15:39:26.115448 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.115415    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:26.117829 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.117810    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 15:39:26.128431 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.128224    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 15:39:26.139183 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.139155    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vt67\" (UniqueName: \"kubernetes.io/projected/9f7ff9d6-945b-462b-b3d4-425473eece9e-kube-api-access-7vt67\") pod \"network-check-target-zkvdb\" (UID: \"9f7ff9d6-945b-462b-b3d4-425473eece9e\") " pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:26.209398 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.209365    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pkvmj\""
Jul 26 15:39:26.217267 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.217225    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8j62j\""
Jul 26 15:39:26.217494 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.217477    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:26.225692 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.225666    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tfwln"
Jul 26 15:39:26.240819 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.240794    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f5rzd"]
Jul 26 15:39:26.552523 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:26.552484    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51eb5653_f3da_4900_8f1d_790a4ec88c7d.slice/crio-2af3aa462dee67191e4dd50e7eb8aaf87ba12980b582ec046968b2b43ca8c1f3 WatchSource:0}: Error finding container 2af3aa462dee67191e4dd50e7eb8aaf87ba12980b582ec046968b2b43ca8c1f3: Status 404 returned error can't find the container with id 2af3aa462dee67191e4dd50e7eb8aaf87ba12980b582ec046968b2b43ca8c1f3
Jul 26 15:39:26.695163 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.695134    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zkvdb"]
Jul 26 15:39:26.700476 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:26.700440    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7ff9d6_945b_462b_b3d4_425473eece9e.slice/crio-f299235bc3dccba86142839a76da650318804e20a06975a5108477a744a7adad WatchSource:0}: Error finding container f299235bc3dccba86142839a76da650318804e20a06975a5108477a744a7adad: Status 404 returned error can't find the container with id f299235bc3dccba86142839a76da650318804e20a06975a5108477a744a7adad
Jul 26 15:39:26.716343 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.716308    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tfwln"]
Jul 26 15:39:26.720286 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:26.720215    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0590d803_5bd9_4739_abf3_13c8bdacc2c0.slice/crio-009fb515365430604e37d622f5a7c22e1700a433b3b72f08bf1ed01a18a8a430 WatchSource:0}: Error finding container 009fb515365430604e37d622f5a7c22e1700a433b3b72f08bf1ed01a18a8a430: Status 404 returned error can't find the container with id 009fb515365430604e37d622f5a7c22e1700a433b3b72f08bf1ed01a18a8a430
Jul 26 15:39:26.724902 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.724793    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2" event={"ID":"1650e062-7149-4008-9c55-c512f46a9606","Type":"ContainerStarted","Data":"ed1afdb9beec0b60fb745d67365aa235442f9bfe17faef1dee1ba8dead5ca200"}
Jul 26 15:39:26.726960 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.726928    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkvdb" event={"ID":"9f7ff9d6-945b-462b-b3d4-425473eece9e","Type":"ContainerStarted","Data":"f299235bc3dccba86142839a76da650318804e20a06975a5108477a744a7adad"}
Jul 26 15:39:26.728105 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:26.728078    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f5rzd" event={"ID":"51eb5653-f3da-4900-8f1d-790a4ec88c7d","Type":"ContainerStarted","Data":"2af3aa462dee67191e4dd50e7eb8aaf87ba12980b582ec046968b2b43ca8c1f3"}
Jul 26 15:39:27.736592 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:27.736498    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tfwln" event={"ID":"0590d803-5bd9-4739-abf3-13c8bdacc2c0","Type":"ContainerStarted","Data":"009fb515365430604e37d622f5a7c22e1700a433b3b72f08bf1ed01a18a8a430"}
Jul 26 15:39:27.740260 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:27.739358    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2" event={"ID":"1650e062-7149-4008-9c55-c512f46a9606","Type":"ContainerStarted","Data":"b5113597c7ad67c3584a2743edb9b367ae484097600c4bb1dcb74316cbd8057a"}
Jul 26 15:39:27.758153 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:27.757963    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-qzcl2" podStartSLOduration=2.356425925 podStartE2EDuration="3.757942798s" totalImagesPullingTime="1.401516873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:27.755618417 +0000 UTC m=+66.939684556" watchObservedRunningTime="2026-07-26 15:39:27.757942798 +0000 UTC m=+66.942008936"
Jul 26 15:39:29.724764 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.724711    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"]
Jul 26 15:39:29.728161 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.728130    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.730491 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.730464    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 26 15:39:29.730616 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.730483    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fqthx\""
Jul 26 15:39:29.730616 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.730471    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 26 15:39:29.739283 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.739262    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"]
Jul 26 15:39:29.742547 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.742526    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"]
Jul 26 15:39:29.742663 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.742647    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.744960 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.744937    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7jwhv\""
Jul 26 15:39:29.745178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.745160    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 26 15:39:29.745397 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.745382    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 26 15:39:29.745662 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.745637    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 26 15:39:29.751258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.751222    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f5zgs"]
Jul 26 15:39:29.755071 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.755051    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.755720 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.755700    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"]
Jul 26 15:39:29.757367 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.757346    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 15:39:29.757462 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.757417    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 15:39:29.757528 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.757515    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 15:39:29.757833 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.757817    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5fmmc\""
Jul 26 15:39:29.841392 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841351    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-textfile\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.841392 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841394    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a028f259-90e1-40ce-b2f2-9c54df73358a-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.841641 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841426    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.841641 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841523    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.841641 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841566    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrbr\" (UniqueName: \"kubernetes.io/projected/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-kube-api-access-nqrbr\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.841641 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841596    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4f8f6cb-3d21-447e-8d59-287225dfc24f-metrics-client-ca\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.841641 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841622    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.841854 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841656    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.841854 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841744    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.841854 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841806    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.841854 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841834    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-tls\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.842062 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841915    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.842062 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841950    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.842062 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.841972    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmdgh\" (UniqueName: \"kubernetes.io/projected/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-api-access-lmdgh\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.842062 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.842030    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-root\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.842062 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.842049    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-wtmp\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.842272 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.842086    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-sys\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.842272 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.842101    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2nfc\" (UniqueName: \"kubernetes.io/projected/a4f8f6cb-3d21-447e-8d59-287225dfc24f-kube-api-access-q2nfc\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.842272 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.842118    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a028f259-90e1-40ce-b2f2-9c54df73358a-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.942957 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.942923    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.942957 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.942971    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrbr\" (UniqueName: \"kubernetes.io/projected/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-kube-api-access-nqrbr\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.943209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.942997    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4f8f6cb-3d21-447e-8d59-287225dfc24f-metrics-client-ca\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943065    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.943209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943107    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.943209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943135    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.943209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943165    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.943209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943189    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-tls\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943577 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943223    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.943577 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:29.943344    2534 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 26 15:39:29.943577 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:29.943398    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-tls podName:a028f259-90e1-40ce-b2f2-9c54df73358a nodeName:}" failed. No retries permitted until 2026-07-26 15:39:30.443382535 +0000 UTC m=+69.627448650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-2dp4s" (UID: "a028f259-90e1-40ce-b2f2-9c54df73358a") : secret "kube-state-metrics-tls" not found
Jul 26 15:39:29.943577 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943460    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.943577 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943496    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmdgh\" (UniqueName: \"kubernetes.io/projected/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-api-access-lmdgh\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.943577 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943557    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-root\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943584    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-wtmp\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943616    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-sys\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943640    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2nfc\" (UniqueName: \"kubernetes.io/projected/a4f8f6cb-3d21-447e-8d59-287225dfc24f-kube-api-access-q2nfc\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943663    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a028f259-90e1-40ce-b2f2-9c54df73358a-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943681    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a4f8f6cb-3d21-447e-8d59-287225dfc24f-metrics-client-ca\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943700    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-accelerators-collector-config\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943710    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-textfile\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943737    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a028f259-90e1-40ce-b2f2-9c54df73358a-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943766    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943830    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-wtmp\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.943889 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.943881    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-root\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.944457 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.944037    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.944457 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.944265    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a028f259-90e1-40ce-b2f2-9c54df73358a-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.944457 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.944304    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.944457 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.944315    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-textfile\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.944457 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.944331    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4f8f6cb-3d21-447e-8d59-287225dfc24f-sys\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.944886 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.944863    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a028f259-90e1-40ce-b2f2-9c54df73358a-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.946163 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.946140    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.946273 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.946182    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.946359 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.946339    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.946695 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.946672    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.946838 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.946820    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a4f8f6cb-3d21-447e-8d59-287225dfc24f-node-exporter-tls\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:29.953735 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.953683    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrbr\" (UniqueName: \"kubernetes.io/projected/7f19d119-1bac-45d6-a580-8b5bb5a1ba80-kube-api-access-nqrbr\") pod \"openshift-state-metrics-b8f49f5c6-vs7wb\" (UID: \"7f19d119-1bac-45d6-a580-8b5bb5a1ba80\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:29.955085 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.955060    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmdgh\" (UniqueName: \"kubernetes.io/projected/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-api-access-lmdgh\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:29.955085 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:29.955079    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2nfc\" (UniqueName: \"kubernetes.io/projected/a4f8f6cb-3d21-447e-8d59-287225dfc24f-kube-api-access-q2nfc\") pod \"node-exporter-f5zgs\" (UID: \"a4f8f6cb-3d21-447e-8d59-287225dfc24f\") " pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:30.040017 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.039917    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"
Jul 26 15:39:30.065392 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.065361    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f5zgs"
Jul 26 15:39:30.446971 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.446941    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:30.449428 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.449400    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a028f259-90e1-40ce-b2f2-9c54df73358a-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-2dp4s\" (UID: \"a028f259-90e1-40ce-b2f2-9c54df73358a\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:30.653312 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.653277    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"
Jul 26 15:39:30.787408 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.787329    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:39:30.791263 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.791242    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.793757 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.793519    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 26 15:39:30.793757 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.793548    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 26 15:39:30.793757 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.793567    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 26 15:39:30.793757 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.793633    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 26 15:39:30.793757 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.793567    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 26 15:39:30.794429 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.794094    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 26 15:39:30.794429 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.794135    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l2qr5\""
Jul 26 15:39:30.794657 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.794606    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 26 15:39:30.794657 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.794615    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 26 15:39:30.794657 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.794636    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 26 15:39:30.804846 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.804150    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:39:30.850703 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850679    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-config-volume\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.850845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850709    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.850845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850734    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.850845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850761    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43a7dde5-457c-4b02-bbeb-ea8219908ffc-config-out\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.850845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850823    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.850845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850841    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43a7dde5-457c-4b02-bbeb-ea8219908ffc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850863    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43a7dde5-457c-4b02-bbeb-ea8219908ffc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850894    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-web-config\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.850932    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.851003    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.851030    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43a7dde5-457c-4b02-bbeb-ea8219908ffc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851245 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.851057    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43a7dde5-457c-4b02-bbeb-ea8219908ffc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.851245 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.851082    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgp89\" (UniqueName: \"kubernetes.io/projected/43a7dde5-457c-4b02-bbeb-ea8219908ffc-kube-api-access-xgp89\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951503 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951464    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951503 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951505    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43a7dde5-457c-4b02-bbeb-ea8219908ffc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951741 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951531    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43a7dde5-457c-4b02-bbeb-ea8219908ffc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951741 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951658    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgp89\" (UniqueName: \"kubernetes.io/projected/43a7dde5-457c-4b02-bbeb-ea8219908ffc-kube-api-access-xgp89\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951741 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951708    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-config-volume\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951741 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951730    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951764    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951789    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43a7dde5-457c-4b02-bbeb-ea8219908ffc-config-out\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951823    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951847    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43a7dde5-457c-4b02-bbeb-ea8219908ffc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951871    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43a7dde5-457c-4b02-bbeb-ea8219908ffc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.951895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951893    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-web-config\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.952181 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.951936    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.952410 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.952387    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/43a7dde5-457c-4b02-bbeb-ea8219908ffc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.952738 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.952493    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43a7dde5-457c-4b02-bbeb-ea8219908ffc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.953227 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.953161    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43a7dde5-457c-4b02-bbeb-ea8219908ffc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.954862 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.954821    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.955471 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.955441    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.955708 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.955686    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.955782 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.955760    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43a7dde5-457c-4b02-bbeb-ea8219908ffc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.956390 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.956366    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43a7dde5-457c-4b02-bbeb-ea8219908ffc-config-out\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.956487 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.956403    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.956562 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.956543    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-web-config\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.957495 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.957467    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-config-volume\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.958419 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.958381    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43a7dde5-457c-4b02-bbeb-ea8219908ffc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:30.960160 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:30.960127    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgp89\" (UniqueName: \"kubernetes.io/projected/43a7dde5-457c-4b02-bbeb-ea8219908ffc-kube-api-access-xgp89\") pod \"alertmanager-main-0\" (UID: \"43a7dde5-457c-4b02-bbeb-ea8219908ffc\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:31.102901 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.102820    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 15:39:31.711313 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.711280    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"]
Jul 26 15:39:31.714906 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.714886    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.717298 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717273    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 26 15:39:31.717399 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717318    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5rven22b7btrd\""
Jul 26 15:39:31.717399 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717284    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 26 15:39:31.717399 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717284    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-sntjs\""
Jul 26 15:39:31.717535 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717478    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 26 15:39:31.717989 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717972    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 26 15:39:31.718041 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.717978    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 26 15:39:31.725097 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.725073    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"]
Jul 26 15:39:31.758670 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.758643    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.758761 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.758687    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-grpc-tls\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.758761 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.758740    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.758878 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.758801    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.758965 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.758933    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-metrics-client-ca\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.759048 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.759015    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-tls\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.759165 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.759071    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c76m\" (UniqueName: \"kubernetes.io/projected/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-kube-api-access-4c76m\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.759165 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.759130    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860395 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860363    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-tls\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860421    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c76m\" (UniqueName: \"kubernetes.io/projected/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-kube-api-access-4c76m\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860460    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860492    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860521    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-grpc-tls\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860559    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860617    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.860842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.860645    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-metrics-client-ca\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.861450 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.861379    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-metrics-client-ca\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.863504 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.863456    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.863695 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.863673    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-grpc-tls\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.863857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.863829    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.863980 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.863962    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.864029 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.863966    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-tls\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.864455 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.864436    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.869138 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:31.869118    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c76m\" (UniqueName: \"kubernetes.io/projected/afa9b5dc-344e-4fa4-9690-d16ab6b9715b-kube-api-access-4c76m\") pod \"thanos-querier-6d54bbcc9b-xq2n4\" (UID: \"afa9b5dc-344e-4fa4-9690-d16ab6b9715b\") " pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:31.978757 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:31.978723    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4f8f6cb_3d21_447e_8d59_287225dfc24f.slice/crio-4c92631522e83efffb2e6ee43056062f1e837f12c892de3851a51f0ef261fd73 WatchSource:0}: Error finding container 4c92631522e83efffb2e6ee43056062f1e837f12c892de3851a51f0ef261fd73: Status 404 returned error can't find the container with id 4c92631522e83efffb2e6ee43056062f1e837f12c892de3851a51f0ef261fd73
Jul 26 15:39:32.024730 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.024539    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:32.175789 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.170812    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 15:39:32.181465 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:32.181436    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43a7dde5_457c_4b02_bbeb_ea8219908ffc.slice/crio-09637da346256800babb17ead6e53cb2ff77bc5ec234a896ba7003da1567c4bd WatchSource:0}: Error finding container 09637da346256800babb17ead6e53cb2ff77bc5ec234a896ba7003da1567c4bd: Status 404 returned error can't find the container with id 09637da346256800babb17ead6e53cb2ff77bc5ec234a896ba7003da1567c4bd
Jul 26 15:39:32.193791 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.193769    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb"]
Jul 26 15:39:32.199760 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:32.199717    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f19d119_1bac_45d6_a580_8b5bb5a1ba80.slice/crio-6ca4b8d43f00ae533c36cb1c2d702bce3a0bf2f3950ac5969a8d06860a94d8b3 WatchSource:0}: Error finding container 6ca4b8d43f00ae533c36cb1c2d702bce3a0bf2f3950ac5969a8d06860a94d8b3: Status 404 returned error can't find the container with id 6ca4b8d43f00ae533c36cb1c2d702bce3a0bf2f3950ac5969a8d06860a94d8b3
Jul 26 15:39:32.206744 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.206653    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s"]
Jul 26 15:39:32.213817 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:32.213786    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda028f259_90e1_40ce_b2f2_9c54df73358a.slice/crio-59cb442a66f3c9c0dafa1b637f8369d502c522a8410756c7420d83db7b3f311f WatchSource:0}: Error finding container 59cb442a66f3c9c0dafa1b637f8369d502c522a8410756c7420d83db7b3f311f: Status 404 returned error can't find the container with id 59cb442a66f3c9c0dafa1b637f8369d502c522a8410756c7420d83db7b3f311f
Jul 26 15:39:32.244607 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.244581    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"]
Jul 26 15:39:32.250518 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:32.250487    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafa9b5dc_344e_4fa4_9690_d16ab6b9715b.slice/crio-01a4a0a1cd94766e31c9ab050161f85d8e187c77fae2853e7910cc443ff09553 WatchSource:0}: Error finding container 01a4a0a1cd94766e31c9ab050161f85d8e187c77fae2853e7910cc443ff09553: Status 404 returned error can't find the container with id 01a4a0a1cd94766e31c9ab050161f85d8e187c77fae2853e7910cc443ff09553
Jul 26 15:39:32.767637 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.767594    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s" event={"ID":"a028f259-90e1-40ce-b2f2-9c54df73358a","Type":"ContainerStarted","Data":"59cb442a66f3c9c0dafa1b637f8369d502c522a8410756c7420d83db7b3f311f"}
Jul 26 15:39:32.769454 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.769420    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"09637da346256800babb17ead6e53cb2ff77bc5ec234a896ba7003da1567c4bd"}
Jul 26 15:39:32.771296 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.771182    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"01a4a0a1cd94766e31c9ab050161f85d8e187c77fae2853e7910cc443ff09553"}
Jul 26 15:39:32.772721 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.772688    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5zgs" event={"ID":"a4f8f6cb-3d21-447e-8d59-287225dfc24f","Type":"ContainerStarted","Data":"4c92631522e83efffb2e6ee43056062f1e837f12c892de3851a51f0ef261fd73"}
Jul 26 15:39:32.775713 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.775652    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8xmfz" event={"ID":"514e1705-96ac-435a-b371-d2fbcfa26998","Type":"ContainerStarted","Data":"8bcff092835b6bef208040685aad02f2b8ae5942f12c93aff2428b0dbfab406b"}
Jul 26 15:39:32.778469 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.778443    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zkvdb" event={"ID":"9f7ff9d6-945b-462b-b3d4-425473eece9e","Type":"ContainerStarted","Data":"1b03067e7c0723470118334b80ebe2f23937fafb09985a3547de9bd5d31b3ff0"}
Jul 26 15:39:32.779875 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.779692    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:39:32.781586 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.781556    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f5rzd" event={"ID":"51eb5653-f3da-4900-8f1d-790a4ec88c7d","Type":"ContainerStarted","Data":"d33bbc559f5721e71deb70f0c252fdce6d121a5da565e1777002eff42849b986"}
Jul 26 15:39:32.784530 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.784388    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb" event={"ID":"7f19d119-1bac-45d6-a580-8b5bb5a1ba80","Type":"ContainerStarted","Data":"a17af2e776775fc1e118c762201cbd6409dfa30e0bf4da8c80a3d294fd7c67a6"}
Jul 26 15:39:32.784530 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.784444    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb" event={"ID":"7f19d119-1bac-45d6-a580-8b5bb5a1ba80","Type":"ContainerStarted","Data":"3795ed3987594542627931c500bd563eaa644d69cb5a4b20638da89a4f2ee3ea"}
Jul 26 15:39:32.784530 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.784470    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb" event={"ID":"7f19d119-1bac-45d6-a580-8b5bb5a1ba80","Type":"ContainerStarted","Data":"6ca4b8d43f00ae533c36cb1c2d702bce3a0bf2f3950ac5969a8d06860a94d8b3"}
Jul 26 15:39:32.787492 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.787466    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tfwln" event={"ID":"0590d803-5bd9-4739-abf3-13c8bdacc2c0","Type":"ContainerStarted","Data":"69327b2af5867e1a5555a5c42e8216bdafca0ab5b295c94307ab559ed940603a"}
Jul 26 15:39:32.787569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.787515    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tfwln" event={"ID":"0590d803-5bd9-4739-abf3-13c8bdacc2c0","Type":"ContainerStarted","Data":"61c75224b73ed19a13ba698c8f67ee8d5803b30a388f043cf1a481f74f12dac8"}
Jul 26 15:39:32.794814 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.794751    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8xmfz" podStartSLOduration=1.6403386599999998 podStartE2EDuration="17.794737762s" totalImagesPullingTime="16.154399102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:39:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:32.794049963 +0000 UTC m=+71.978116103" watchObservedRunningTime="2026-07-26 15:39:32.794737762 +0000 UTC m=+71.978803898"
Jul 26 15:39:32.811304 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.811252    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tfwln" podStartSLOduration=66.481463402 podStartE2EDuration="1m11.811217957s" totalImagesPullingTime="5.329754555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:32.810619421 +0000 UTC m=+71.994685559" watchObservedRunningTime="2026-07-26 15:39:32.811217957 +0000 UTC m=+71.995284093"
Jul 26 15:39:32.824973 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.824890    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f5rzd" podStartSLOduration=66.403749769 podStartE2EDuration="1m11.824879441s" totalImagesPullingTime="5.421129672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:32.824832759 +0000 UTC m=+72.008898898" watchObservedRunningTime="2026-07-26 15:39:32.824879441 +0000 UTC m=+72.008945579"
Jul 26 15:39:32.840639 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:32.840595    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zkvdb" podStartSLOduration=66.569161007 podStartE2EDuration="1m11.840580288s" totalImagesPullingTime="5.271419281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:38:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:32.839833543 +0000 UTC m=+72.023899679" watchObservedRunningTime="2026-07-26 15:39:32.840580288 +0000 UTC m=+72.024646425"
Jul 26 15:39:33.712293 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:33.712264    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6tbt2"
Jul 26 15:39:34.044831 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.044753    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-868445f98d-q5ccn"]
Jul 26 15:39:34.048380 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.048356    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.051064 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.051031    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 26 15:39:34.052265 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.052225    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 26 15:39:34.056482 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.052628    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7hwtf\""
Jul 26 15:39:34.056482 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.052700    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 26 15:39:34.056482 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.053090    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 26 15:39:34.056954 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.056838    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-82ssmgqn7fhn3\""
Jul 26 15:39:34.056954 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.056878    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-868445f98d-q5ccn"]
Jul 26 15:39:34.079660 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079628    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-secret-metrics-server-tls\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.079772 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079689    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kklxf\" (UniqueName: \"kubernetes.io/projected/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-kube-api-access-kklxf\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.079772 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079748    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.079877 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079787    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-secret-metrics-server-client-certs\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.079877 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079848    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-audit-log\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.079960 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079872    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-metrics-server-audit-profiles\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.079960 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.079893    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-client-ca-bundle\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.180983 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.180944    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-secret-metrics-server-client-certs\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.180998    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-audit-log\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.181030    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-metrics-server-audit-profiles\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.181057    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-client-ca-bundle\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.181114    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-secret-metrics-server-tls\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.181146    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kklxf\" (UniqueName: \"kubernetes.io/projected/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-kube-api-access-kklxf\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181437 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.181217    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.181883 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.181831    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-audit-log\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.182169 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.182126    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.182446 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.182399    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-metrics-server-audit-profiles\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.183937 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.183910    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-secret-metrics-server-tls\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.184722 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.184698    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-secret-metrics-server-client-certs\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.186186 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.186164    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-client-ca-bundle\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.189202 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.189184    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kklxf\" (UniqueName: \"kubernetes.io/projected/f2fd3614-cc28-4a8d-9a64-a0318f058bdd-kube-api-access-kklxf\") pod \"metrics-server-868445f98d-q5ccn\" (UID: \"f2fd3614-cc28-4a8d-9a64-a0318f058bdd\") " pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.365502 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.365421    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:34.526345 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.526050    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d5bd67dc-qdd94"]
Jul 26 15:39:34.531287 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.530618    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.532922 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.532856    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 26 15:39:34.533067 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.533048    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 26 15:39:34.534166 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.533996    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 26 15:39:34.534166 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.534073    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rr5w4\""
Jul 26 15:39:34.534365 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.534268    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 26 15:39:34.534418 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.534366    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 26 15:39:34.534530 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.534478    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 26 15:39:34.534638 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.534607    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 26 15:39:34.539441 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.539138    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"]
Jul 26 15:39:34.544863 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.544355    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:34.545743 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.545708    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 26 15:39:34.545978 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.545916    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5bd67dc-qdd94"]
Jul 26 15:39:34.547358 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.547302    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6dnxw\""
Jul 26 15:39:34.548084 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.547542    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 26 15:39:34.561506 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.560894    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"]
Jul 26 15:39:34.584987 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.584952    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v22x9\" (UniqueName: \"kubernetes.io/projected/f7f98ca9-91d9-4a94-a414-e75a90d9449a-kube-api-access-v22x9\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:34.585078 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585044    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-oauth-config\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.585147 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585081    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-config\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.585147 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585117    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-service-ca\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.585269 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585150    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vtk9\" (UniqueName: \"kubernetes.io/projected/261ea5ff-66b3-4f18-95e8-d0303ff642cf-kube-api-access-6vtk9\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.585269 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585189    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-serving-cert\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.585269 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585222    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7f98ca9-91d9-4a94-a414-e75a90d9449a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:34.585432 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585282    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-oauth-serving-cert\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.585432 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.585351    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-trusted-ca-bundle\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.604872 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.604813    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-868445f98d-q5ccn"]
Jul 26 15:39:34.686006 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.685965    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-oauth-config\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686006 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686003    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-config\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686205 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686033    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-service-ca\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686205 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686062    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vtk9\" (UniqueName: \"kubernetes.io/projected/261ea5ff-66b3-4f18-95e8-d0303ff642cf-kube-api-access-6vtk9\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686205 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686094    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-serving-cert\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686509    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7f98ca9-91d9-4a94-a414-e75a90d9449a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686573    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-oauth-serving-cert\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686622    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-trusted-ca-bundle\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:34.686630    2534 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686664    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v22x9\" (UniqueName: \"kubernetes.io/projected/f7f98ca9-91d9-4a94-a414-e75a90d9449a-kube-api-access-v22x9\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:39:34.686703    2534 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7f98ca9-91d9-4a94-a414-e75a90d9449a-monitoring-plugin-cert podName:f7f98ca9-91d9-4a94-a414-e75a90d9449a nodeName:}" failed. No retries permitted until 2026-07-26 15:39:35.18668253 +0000 UTC m=+74.370748694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f7f98ca9-91d9-4a94-a414-e75a90d9449a-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-8vcj4" (UID: "f7f98ca9-91d9-4a94-a414-e75a90d9449a") : secret "monitoring-plugin-cert" not found
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686885    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-service-ca\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.686931 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.686885    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-config\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.687487 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.687464    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-oauth-serving-cert\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.687587 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.687562    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-trusted-ca-bundle\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.689419 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.689397    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-serving-cert\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.689678 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.689656    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-oauth-config\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.697541 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.697470    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vtk9\" (UniqueName: \"kubernetes.io/projected/261ea5ff-66b3-4f18-95e8-d0303ff642cf-kube-api-access-6vtk9\") pod \"console-5d5bd67dc-qdd94\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") " pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.698250 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.698202    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v22x9\" (UniqueName: \"kubernetes.io/projected/f7f98ca9-91d9-4a94-a414-e75a90d9449a-kube-api-access-v22x9\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:34.798215 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.798178    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb" event={"ID":"7f19d119-1bac-45d6-a580-8b5bb5a1ba80","Type":"ContainerStarted","Data":"e9203bbd34e5e4d708b5423c4c716f52c5b38b3db78f2d2f90c6371cedc6317b"}
Jul 26 15:39:34.800392 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.800363    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s" event={"ID":"a028f259-90e1-40ce-b2f2-9c54df73358a","Type":"ContainerStarted","Data":"61500cda6c318b6619d45ebef8c582b08056eda44e50e6c05ef719e57b70af72"}
Jul 26 15:39:34.800475 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.800429    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s" event={"ID":"a028f259-90e1-40ce-b2f2-9c54df73358a","Type":"ContainerStarted","Data":"ca11751781a3c519e2962becbbd97add68646f569bdec712b0161a734f4ba7d3"}
Jul 26 15:39:34.802899 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.802769    2534 generic.go:356] "Generic (PLEG): container finished" podID="43a7dde5-457c-4b02-bbeb-ea8219908ffc" containerID="21688d899f191b8666a013f831a0c15b3444f36e6667a2c0581d6b63bf947bb2" exitCode=0
Jul 26 15:39:34.802899 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.802867    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerDied","Data":"21688d899f191b8666a013f831a0c15b3444f36e6667a2c0581d6b63bf947bb2"}
Jul 26 15:39:34.805598 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.805543    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"ccbaee75ac80d2fb78926d2602e4ae77b013f4bf1d55e117003fcac0a3ec9a50"}
Jul 26 15:39:34.805693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.805617    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"42d8c2a71d906ab52434813a59e2d20b023e9b89be97fad911213a461d7b3dd1"}
Jul 26 15:39:34.807880 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.807792    2534 generic.go:356] "Generic (PLEG): container finished" podID="a4f8f6cb-3d21-447e-8d59-287225dfc24f" containerID="62526c1b5415511aa910efa03859db0bb7bbf04668f05a7621821fe84facc86c" exitCode=0
Jul 26 15:39:34.807880 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.807831    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5zgs" event={"ID":"a4f8f6cb-3d21-447e-8d59-287225dfc24f","Type":"ContainerDied","Data":"62526c1b5415511aa910efa03859db0bb7bbf04668f05a7621821fe84facc86c"}
Jul 26 15:39:34.809428 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.809376    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn" event={"ID":"f2fd3614-cc28-4a8d-9a64-a0318f058bdd","Type":"ContainerStarted","Data":"a0fd718416701670b271e077546bca147cd5f4c51416831a8a41364cd992cfd0"}
Jul 26 15:39:34.818422 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.818357    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vs7wb" podStartSLOduration=3.712371877 podStartE2EDuration="5.818338691s" totalImagesPullingTime="2.105966814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:34.81689438 +0000 UTC m=+74.000960541" watchObservedRunningTime="2026-07-26 15:39:34.818338691 +0000 UTC m=+74.002404831"
Jul 26 15:39:34.878367 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.878344    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:34.947740 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.947711    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"]
Jul 26 15:39:34.951516 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.951493    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.953843 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.953818    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 26 15:39:34.953956 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.953873    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-p7hlv\""
Jul 26 15:39:34.954029 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.953998    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 26 15:39:34.954095 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.954079    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 26 15:39:34.954549 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.954529    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 26 15:39:34.954881 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.954865    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 26 15:39:34.959653 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.959632    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 26 15:39:34.968220 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.968192    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"]
Jul 26 15:39:34.990852 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.990817    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-telemeter-client-tls\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.990997 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.990866    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-federate-client-tls\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.990997 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.990922    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.990997 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.990976    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-serving-certs-ca-bundle\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.991160 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.991012    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.991160 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.991072    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-metrics-client-ca\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.991160 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.991144    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-secret-telemeter-client\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:34.991335 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:34.991185    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtxsc\" (UniqueName: \"kubernetes.io/projected/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-kube-api-access-mtxsc\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.026060 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.026035    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d5bd67dc-qdd94"]
Jul 26 15:39:35.028386 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:35.028359    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261ea5ff_66b3_4f18_95e8_d0303ff642cf.slice/crio-73942f6e596dedc07db175b122cc4b17a1f5568200474af0e7f031c9da0e5bed WatchSource:0}: Error finding container 73942f6e596dedc07db175b122cc4b17a1f5568200474af0e7f031c9da0e5bed: Status 404 returned error can't find the container with id 73942f6e596dedc07db175b122cc4b17a1f5568200474af0e7f031c9da0e5bed
Jul 26 15:39:35.092254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092195    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtxsc\" (UniqueName: \"kubernetes.io/projected/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-kube-api-access-mtxsc\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092254    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-telemeter-client-tls\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092468 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092283    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-federate-client-tls\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092468 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092310    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092468 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092329    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-serving-certs-ca-bundle\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092468 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092358    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092468 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092437    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-metrics-client-ca\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.092707 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.092508    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-secret-telemeter-client\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.093270 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.093180    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-serving-certs-ca-bundle\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.093270 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.093261    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-metrics-client-ca\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.093643 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.093614    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.094990 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.094964    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.095076 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.094999    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-federate-client-tls\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.095076 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.095028    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-secret-telemeter-client\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.095158 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.095128    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-telemeter-client-tls\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.099730 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.099707    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtxsc\" (UniqueName: \"kubernetes.io/projected/ab6c163d-8b6d-453d-b48a-f51ee53ff14e-kube-api-access-mtxsc\") pod \"telemeter-client-7cbd46bd9c-g5bqr\" (UID: \"ab6c163d-8b6d-453d-b48a-f51ee53ff14e\") " pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.193925 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.193849    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7f98ca9-91d9-4a94-a414-e75a90d9449a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:35.196324 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.196302    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f7f98ca9-91d9-4a94-a414-e75a90d9449a-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-8vcj4\" (UID: \"f7f98ca9-91d9-4a94-a414-e75a90d9449a\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:35.264612 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.264551    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"
Jul 26 15:39:35.451720 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.451630    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr"]
Jul 26 15:39:35.487813 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.487782    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:35.683065 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.683030    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"]
Jul 26 15:39:35.823067 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.822939    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5bd67dc-qdd94" event={"ID":"261ea5ff-66b3-4f18-95e8-d0303ff642cf","Type":"ContainerStarted","Data":"73942f6e596dedc07db175b122cc4b17a1f5568200474af0e7f031c9da0e5bed"}
Jul 26 15:39:35.827741 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.827701    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s" event={"ID":"a028f259-90e1-40ce-b2f2-9c54df73358a","Type":"ContainerStarted","Data":"c4a39821b34ab695c23eca9422f6f2f3b3bee6f90eb1dea34976b2e8ebb661f0"}
Jul 26 15:39:35.851919 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.851879    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"e0f096a369e51881fffc94659f660ca4419ae36c2833cf6cec8675868ede2a0d"}
Jul 26 15:39:35.856272 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.856167    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5zgs" event={"ID":"a4f8f6cb-3d21-447e-8d59-287225dfc24f","Type":"ContainerStarted","Data":"cc34fdad52e4582dce64a67910ba916878808604bda1af4fe18dc0277c21f7c3"}
Jul 26 15:39:35.856272 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.856218    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f5zgs" event={"ID":"a4f8f6cb-3d21-447e-8d59-287225dfc24f","Type":"ContainerStarted","Data":"2ed6e8c6a1b46d7ed27e57dbc11d2ffa7d6fb6e16355c787ede325f8d048cbfb"}
Jul 26 15:39:35.862275 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.860822    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr" event={"ID":"ab6c163d-8b6d-453d-b48a-f51ee53ff14e","Type":"ContainerStarted","Data":"a1b9384960802a5e663c06c96a62b014d7f8ea45e1a041d7d717262aeb7a4139"}
Jul 26 15:39:35.863310 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.863264    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-2dp4s" podStartSLOduration=4.641943034 podStartE2EDuration="6.863213236s" totalImagesPullingTime="2.221270202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:35.861299889 +0000 UTC m=+75.045366027" watchObservedRunningTime="2026-07-26 15:39:35.863213236 +0000 UTC m=+75.047279375"
Jul 26 15:39:35.883496 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:35.883302    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f5zgs" podStartSLOduration=5.864943293 podStartE2EDuration="6.883280967s" totalImagesPullingTime="909.695659ms" totalInitContainerRuntime="108.642015ms" isStatefulPod=true podCreationTimestamp="2026-07-26 15:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:35.883219718 +0000 UTC m=+75.067285856" watchObservedRunningTime="2026-07-26 15:39:35.883280967 +0000 UTC m=+75.067347107"
Jul 26 15:39:35.929576 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:39:35.929530    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f98ca9_91d9_4a94_a414_e75a90d9449a.slice/crio-564115625bb459d4541d7dd36cb212ab33dae01e4486f56b91adf726173c3a30 WatchSource:0}: Error finding container 564115625bb459d4541d7dd36cb212ab33dae01e4486f56b91adf726173c3a30: Status 404 returned error can't find the container with id 564115625bb459d4541d7dd36cb212ab33dae01e4486f56b91adf726173c3a30
Jul 26 15:39:36.049324 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.048785    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:39:36.053742 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.053707    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.056787    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.056992    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.056997    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2vgt45fjqkbc5\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.057330    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.057388    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.057644    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.057650    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.057846    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.057908    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.058050    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.058083    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4thxt\""
Jul 26 15:39:36.058826 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.058252    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 26 15:39:36.059405 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.059335    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 26 15:39:36.064817 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.064793    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 26 15:39:36.065971 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.065779    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 26 15:39:36.067438 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.067420    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 26 15:39:36.072126 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.072102    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104539    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104583    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104611    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj9q9\" (UniqueName: \"kubernetes.io/projected/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-kube-api-access-bj9q9\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104644    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104678    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104725    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104750    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104782    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104815    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104845    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104874    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104903    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104946    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.104971    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.105001    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.105571 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.105024    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.106445 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.105048    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.106445 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.105078    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.106445 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.105123    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.106445 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.105162    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-config\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206059 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206023    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206076    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206108    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206150    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206176    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206204    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206254 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206244    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206267    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206298    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206366    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206403    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-config\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206438    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206464    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206536 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206535    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj9q9\" (UniqueName: \"kubernetes.io/projected/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-kube-api-access-bj9q9\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206611    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206644    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206697    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206722    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206753    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.206857 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.206788    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.208374    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.208550    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.208883    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.209053    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.209144    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.209746    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.210134    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.212042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.211977    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.220306 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.219617    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.220306 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.219938    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.220306 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.219990    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-config-out\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.220306 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.220251    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.220619 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.220356    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.220619 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.220441    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.221275 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.220846    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-config\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.221275 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.221218    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.221424 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.221283    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.222276 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.222202    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-web-config\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.224726 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.224682    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.225270 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.225249    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj9q9\" (UniqueName: \"kubernetes.io/projected/15d65b7b-3e3f-48a3-ad77-b4aa86c994c0-kube-api-access-bj9q9\") pod \"prometheus-k8s-0\" (UID: \"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.380054 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.379950    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:36.868855 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.868777    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4" event={"ID":"f7f98ca9-91d9-4a94-a414-e75a90d9449a","Type":"ContainerStarted","Data":"564115625bb459d4541d7dd36cb212ab33dae01e4486f56b91adf726173c3a30"}
Jul 26 15:39:36.876176 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:36.874709    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"9cc7252d57280a6455a4dcff90f71ae9d568d11dd81f4eb417e022547e414ce6"}
Jul 26 15:39:39.551104 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.550943    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 15:39:39.894728 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.894667    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr" event={"ID":"ab6c163d-8b6d-453d-b48a-f51ee53ff14e","Type":"ContainerStarted","Data":"d961a35ad2dd9be2b5b9e09b2f2dde8952679a90e3462609b2779b029cde4c79"}
Jul 26 15:39:39.894840 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.894770    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr" event={"ID":"ab6c163d-8b6d-453d-b48a-f51ee53ff14e","Type":"ContainerStarted","Data":"eafd7b2b02095cd2241b82c76baebb71c0114eb44c1685aeedb2bd2176c73d3f"}
Jul 26 15:39:39.894840 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.894801    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr" event={"ID":"ab6c163d-8b6d-453d-b48a-f51ee53ff14e","Type":"ContainerStarted","Data":"63af76fc4768f5aa7f5d9867d33ea2923d3b118d2be5e00d95293dbabea8922e"}
Jul 26 15:39:39.896520 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.896497    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5bd67dc-qdd94" event={"ID":"261ea5ff-66b3-4f18-95e8-d0303ff642cf","Type":"ContainerStarted","Data":"69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95"}
Jul 26 15:39:39.898962 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.898509    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4" event={"ID":"f7f98ca9-91d9-4a94-a414-e75a90d9449a","Type":"ContainerStarted","Data":"d5af330b14d693727ab3bdc43e69bfc1f802df7cf9c674d740a8fce115b9e8db"}
Jul 26 15:39:39.899143 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.899101    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:39.904693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.904447    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"a02a03370357220ba613bde3f62f73673c94e7b91d37ff223e8d798b6c525f42"}
Jul 26 15:39:39.904693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.904514    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"aa90615d13d44bf6c9178e1a92c96bbd51074f88d342bb7b87039efc8ddf016d"}
Jul 26 15:39:39.904693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.904556    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"787da83d7cce80281d045efc37aef791c76b3a67db88f193c43968b2162baa2e"}
Jul 26 15:39:39.904693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.904596    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"3be22bbb804390f4290972adff60b473e5054e01c753d2d3d63fc13bb3a3b3de"}
Jul 26 15:39:39.904693 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.904633    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"4e97bb2696181ef9f616ef637d82b27881aba9b76108a87f6a7b6995f8eb7cb9"}
Jul 26 15:39:39.905165 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.905135    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4"
Jul 26 15:39:39.906339 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.906317    2534 generic.go:356] "Generic (PLEG): container finished" podID="15d65b7b-3e3f-48a3-ad77-b4aa86c994c0" containerID="8272ab008cb6de837edf35a549d6d61610a6d677f558aff3d909541f4ce51bed" exitCode=0
Jul 26 15:39:39.906521 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.906504    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerDied","Data":"8272ab008cb6de837edf35a549d6d61610a6d677f558aff3d909541f4ce51bed"}
Jul 26 15:39:39.906661 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.906643    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"b6af111cbb191a5b72b0a44c82abeb4070a4aabcda47c7e2b58ef4db51f862d3"}
Jul 26 15:39:39.911881 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.911855    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"14bebbe4314367e7f1169ae1046939c2abc47f20e7198e48d6ad9c3ec633f7ca"}
Jul 26 15:39:39.911977 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.911921    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" event={"ID":"afa9b5dc-344e-4fa4-9690-d16ab6b9715b","Type":"ContainerStarted","Data":"b88a8e15bd2cb047bd1c8842e0129d7bb0da4a5b0dfa86b94d1adf0bcac6db08"}
Jul 26 15:39:39.912075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.912058    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:39.914002 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.913830    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn" event={"ID":"f2fd3614-cc28-4a8d-9a64-a0318f058bdd","Type":"ContainerStarted","Data":"2402a00751b525d3e616f53f687d92b01a8caf95353c291b193d86bb731d6c91"}
Jul 26 15:39:39.917980 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.917938    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7cbd46bd9c-g5bqr" podStartSLOduration=1.9731386469999999 podStartE2EDuration="5.917921698s" totalImagesPullingTime="3.944783051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:39.916477453 +0000 UTC m=+79.100543593" watchObservedRunningTime="2026-07-26 15:39:39.917921698 +0000 UTC m=+79.101987834"
Jul 26 15:39:39.919439 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.919420    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4"
Jul 26 15:39:39.937253 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.936909    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d54bbcc9b-xq2n4" podStartSLOduration=5.550380584 podStartE2EDuration="8.93689097s" totalImagesPullingTime="3.386510386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:39.936132894 +0000 UTC m=+79.120199032" watchObservedRunningTime="2026-07-26 15:39:39.93689097 +0000 UTC m=+79.120957129"
Jul 26 15:39:39.951020 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.950975    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-8vcj4" podStartSLOduration=2.473562727 podStartE2EDuration="5.950963388s" totalImagesPullingTime="3.477400661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:39.949197219 +0000 UTC m=+79.133263356" watchObservedRunningTime="2026-07-26 15:39:39.950963388 +0000 UTC m=+79.135029503"
Jul 26 15:39:39.968010 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:39.967948    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d5bd67dc-qdd94" podStartSLOduration=1.584036684 podStartE2EDuration="5.967929096s" totalImagesPullingTime="4.383892412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:39.966630894 +0000 UTC m=+79.150697035" watchObservedRunningTime="2026-07-26 15:39:39.967929096 +0000 UTC m=+79.151995233"
Jul 26 15:39:40.008376 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:40.008322    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn" podStartSLOduration=1.230455961 podStartE2EDuration="6.00830073s" totalImagesPullingTime="4.777844769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:40.006810507 +0000 UTC m=+79.190876644" watchObservedRunningTime="2026-07-26 15:39:40.00830073 +0000 UTC m=+79.192366866"
Jul 26 15:39:40.850516 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:40.850478    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5bd67dc-qdd94"]
Jul 26 15:39:40.923443 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:40.923327    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"43a7dde5-457c-4b02-bbeb-ea8219908ffc","Type":"ContainerStarted","Data":"1900dad16b54d0d84dfd9816e6c3aad74dae170286624357285e2234488d8427"}
Jul 26 15:39:40.950042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:40.949985    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.067788812 podStartE2EDuration="10.949968466s" totalImagesPullingTime="6.851710825s" totalInitContainerRuntime="30.468829ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:40.948830291 +0000 UTC m=+80.132896429" watchObservedRunningTime="2026-07-26 15:39:40.949968466 +0000 UTC m=+80.134034602"
Jul 26 15:39:43.941708 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.941676    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"635d69ba7eb72c9aab2866bdeca745091c2929fe0653e9b9c944705568544887"}
Jul 26 15:39:43.942075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.941751    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"a0043a83c3d3d57bce405096c1133ad89ea0afd8605d0af59e4dddef8fd56f18"}
Jul 26 15:39:43.942075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.941778    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"ac965ec48a6e21bc26d86dd9d3d8918c60e63551f710697cbc217e61105002bd"}
Jul 26 15:39:43.942075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.941806    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"4818232a55b8fe295925e1d245956b2196ea63c5e9adf1b47b8dea5c6d2195d1"}
Jul 26 15:39:43.942075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.941831    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"215ced9bb73e7a4cbc0c3f55fddb2b163dc1c774c3a18b8d615f15dff06f9571"}
Jul 26 15:39:43.942075 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.941857    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"15d65b7b-3e3f-48a3-ad77-b4aa86c994c0","Type":"ContainerStarted","Data":"a4b52b25d4e77f97642b1955831205a5caf44ca3a3091c9d275b965f8fb669d7"}
Jul 26 15:39:43.971366 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:43.971312    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.942401951 podStartE2EDuration="7.971295272s" totalImagesPullingTime="2.998345505s" totalInitContainerRuntime="30.547816ms" isStatefulPod=false podCreationTimestamp="2026-07-26 15:39:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:39:43.968432133 +0000 UTC m=+83.152498269" watchObservedRunningTime="2026-07-26 15:39:43.971295272 +0000 UTC m=+83.155361408"
Jul 26 15:39:44.879226 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:44.879189    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:39:46.381340 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:46.381287    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:39:54.366501 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:54.366458    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:39:54.366954 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:39:54.366523    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:40:04.812694 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:04.812659    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zkvdb"
Jul 26 15:40:06.953129 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:06.953081    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d5bd67dc-qdd94" podUID="261ea5ff-66b3-4f18-95e8-d0303ff642cf" containerName="console" containerID="cri-o://69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95" gracePeriod=15
Jul 26 15:40:07.725423 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.725390    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:40:07.817660 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817633    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-serving-cert\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.817853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817678    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-oauth-config\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.817853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817696    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vtk9\" (UniqueName: \"kubernetes.io/projected/261ea5ff-66b3-4f18-95e8-d0303ff642cf-kube-api-access-6vtk9\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.817853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817731    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-config\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.817853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817749    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-oauth-serving-cert\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.817853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817805    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-service-ca\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.817853 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.817837    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-trusted-ca-bundle\") pod \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\" (UID: \"261ea5ff-66b3-4f18-95e8-d0303ff642cf\") "
Jul 26 15:40:07.818339 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.818310    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-config" (OuterVolumeSpecName: "console-config") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:40:07.818339 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.818326    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-service-ca" (OuterVolumeSpecName: "service-ca") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:40:07.818339 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.818334    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:40:07.818517 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.818318    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 15:40:07.820027 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.820003    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:40:07.820409 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.820378    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261ea5ff-66b3-4f18-95e8-d0303ff642cf-kube-api-access-6vtk9" (OuterVolumeSpecName: "kube-api-access-6vtk9") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "kube-api-access-6vtk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:40:07.820522 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.820411    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "261ea5ff-66b3-4f18-95e8-d0303ff642cf" (UID: "261ea5ff-66b3-4f18-95e8-d0303ff642cf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.918967    2534 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-service-ca\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.918997    2534 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-trusted-ca-bundle\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.919007    2534 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-serving-cert\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.919018    2534 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-oauth-config\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.919026    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vtk9\" (UniqueName: \"kubernetes.io/projected/261ea5ff-66b3-4f18-95e8-d0303ff642cf-kube-api-access-6vtk9\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.919035    2534 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-console-config\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:07.919057 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:07.919043    2534 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/261ea5ff-66b3-4f18-95e8-d0303ff642cf-oauth-serving-cert\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:40:08.042410 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.042378    2534 generic.go:356] "Generic (PLEG): container finished" podID="261ea5ff-66b3-4f18-95e8-d0303ff642cf" containerID="69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95" exitCode=0
Jul 26 15:40:08.042920 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.042419    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d5bd67dc-qdd94"
Jul 26 15:40:08.042920 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.042468    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5bd67dc-qdd94" event={"ID":"261ea5ff-66b3-4f18-95e8-d0303ff642cf","Type":"ContainerDied","Data":"69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95"}
Jul 26 15:40:08.042920 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.042528    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d5bd67dc-qdd94" event={"ID":"261ea5ff-66b3-4f18-95e8-d0303ff642cf","Type":"ContainerDied","Data":"73942f6e596dedc07db175b122cc4b17a1f5568200474af0e7f031c9da0e5bed"}
Jul 26 15:40:08.042920 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.042560    2534 scope.go:117] "RemoveContainer" containerID="69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95"
Jul 26 15:40:08.051825 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.051800    2534 scope.go:117] "RemoveContainer" containerID="69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95"
Jul 26 15:40:08.052098 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:40:08.052077    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95\": container with ID starting with 69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95 not found: ID does not exist" containerID="69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95"
Jul 26 15:40:08.052166 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.052111    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95"} err="failed to get container status \"69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95\": rpc error: code = NotFound desc = could not find container \"69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95\": container with ID starting with 69066f9a8927953326e3a8f66898f5b9edba2505c696f56bd9aa407cfd54bf95 not found: ID does not exist"
Jul 26 15:40:08.062743 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.062719    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d5bd67dc-qdd94"]
Jul 26 15:40:08.066523 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:08.066500    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d5bd67dc-qdd94"]
Jul 26 15:40:09.296823 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:09.296790    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261ea5ff-66b3-4f18-95e8-d0303ff642cf" path="/var/lib/kubelet/pods/261ea5ff-66b3-4f18-95e8-d0303ff642cf/volumes"
Jul 26 15:40:14.371624 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:14.371592    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:40:14.375500 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:14.375479    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-868445f98d-q5ccn"
Jul 26 15:40:36.381325 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:36.381288    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:40:36.399679 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:36.399655    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:40:37.180974 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:40:37.180950    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 15:42:12.013878 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.013840    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-crvfv"]
Jul 26 15:42:12.014379 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.014165    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261ea5ff-66b3-4f18-95e8-d0303ff642cf" containerName="console"
Jul 26 15:42:12.014379 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.014175    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="261ea5ff-66b3-4f18-95e8-d0303ff642cf" containerName="console"
Jul 26 15:42:12.014379 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.014248    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="261ea5ff-66b3-4f18-95e8-d0303ff642cf" containerName="console"
Jul 26 15:42:12.017348 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.017332    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.019865 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.019844    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-xnj57\""
Jul 26 15:42:12.020003 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.019917    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 26 15:42:12.020732 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.020707    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 26 15:42:12.027305 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.027284    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-crvfv"]
Jul 26 15:42:12.104198 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.104174    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dee3d7e-c67e-484e-97ea-67917717ecee-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-crvfv\" (UID: \"8dee3d7e-c67e-484e-97ea-67917717ecee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.104348 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.104224    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxk2l\" (UniqueName: \"kubernetes.io/projected/8dee3d7e-c67e-484e-97ea-67917717ecee-kube-api-access-vxk2l\") pod \"cert-manager-webhook-6486ffd997-crvfv\" (UID: \"8dee3d7e-c67e-484e-97ea-67917717ecee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.205284 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.205228    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dee3d7e-c67e-484e-97ea-67917717ecee-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-crvfv\" (UID: \"8dee3d7e-c67e-484e-97ea-67917717ecee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.205436 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.205317    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxk2l\" (UniqueName: \"kubernetes.io/projected/8dee3d7e-c67e-484e-97ea-67917717ecee-kube-api-access-vxk2l\") pod \"cert-manager-webhook-6486ffd997-crvfv\" (UID: \"8dee3d7e-c67e-484e-97ea-67917717ecee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.214464 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.214442    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxk2l\" (UniqueName: \"kubernetes.io/projected/8dee3d7e-c67e-484e-97ea-67917717ecee-kube-api-access-vxk2l\") pod \"cert-manager-webhook-6486ffd997-crvfv\" (UID: \"8dee3d7e-c67e-484e-97ea-67917717ecee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.214903 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.214880    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8dee3d7e-c67e-484e-97ea-67917717ecee-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-crvfv\" (UID: \"8dee3d7e-c67e-484e-97ea-67917717ecee\") " pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.342086 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.342009    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:12.469351 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.469268    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-crvfv"]
Jul 26 15:42:12.471618 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:42:12.471589    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8dee3d7e_c67e_484e_97ea_67917717ecee.slice/crio-b4f8ad2c78005f1523b934f822df455af542f288705823a94dbe51d2335de4e2 WatchSource:0}: Error finding container b4f8ad2c78005f1523b934f822df455af542f288705823a94dbe51d2335de4e2: Status 404 returned error can't find the container with id b4f8ad2c78005f1523b934f822df455af542f288705823a94dbe51d2335de4e2
Jul 26 15:42:12.531454 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:12.531423    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv" event={"ID":"8dee3d7e-c67e-484e-97ea-67917717ecee","Type":"ContainerStarted","Data":"b4f8ad2c78005f1523b934f822df455af542f288705823a94dbe51d2335de4e2"}
Jul 26 15:42:14.308432 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.308099    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"]
Jul 26 15:42:14.311924 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.311891    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.314197 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.314144    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xqq7q\""
Jul 26 15:42:14.319899 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.319853    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"]
Jul 26 15:42:14.436618 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.436582    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08469263-faa4-4324-893f-a9d6e79658cf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4j9md\" (UID: \"08469263-faa4-4324-893f-a9d6e79658cf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.436811 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.436631    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqswk\" (UniqueName: \"kubernetes.io/projected/08469263-faa4-4324-893f-a9d6e79658cf-kube-api-access-kqswk\") pod \"cert-manager-cainjector-559cdd8c87-4j9md\" (UID: \"08469263-faa4-4324-893f-a9d6e79658cf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.537222 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.537183    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08469263-faa4-4324-893f-a9d6e79658cf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4j9md\" (UID: \"08469263-faa4-4324-893f-a9d6e79658cf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.537421 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.537227    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqswk\" (UniqueName: \"kubernetes.io/projected/08469263-faa4-4324-893f-a9d6e79658cf-kube-api-access-kqswk\") pod \"cert-manager-cainjector-559cdd8c87-4j9md\" (UID: \"08469263-faa4-4324-893f-a9d6e79658cf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.546433 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.546407    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqswk\" (UniqueName: \"kubernetes.io/projected/08469263-faa4-4324-893f-a9d6e79658cf-kube-api-access-kqswk\") pod \"cert-manager-cainjector-559cdd8c87-4j9md\" (UID: \"08469263-faa4-4324-893f-a9d6e79658cf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.546568 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.546486    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08469263-faa4-4324-893f-a9d6e79658cf-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4j9md\" (UID: \"08469263-faa4-4324-893f-a9d6e79658cf\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:14.626437 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:14.626356    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"
Jul 26 15:42:16.301796 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.301775    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4j9md"]
Jul 26 15:42:16.303673 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:42:16.303651    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08469263_faa4_4324_893f_a9d6e79658cf.slice/crio-00db919c4ba4b72482d9b1ba20c9362b54bf8a009c7d5beaf65bc7b30a46631f WatchSource:0}: Error finding container 00db919c4ba4b72482d9b1ba20c9362b54bf8a009c7d5beaf65bc7b30a46631f: Status 404 returned error can't find the container with id 00db919c4ba4b72482d9b1ba20c9362b54bf8a009c7d5beaf65bc7b30a46631f
Jul 26 15:42:16.551772 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.551733    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md" event={"ID":"08469263-faa4-4324-893f-a9d6e79658cf","Type":"ContainerStarted","Data":"a460025fdd4b461ecabd4b2fa928107b1f0d50430a814d745b2aba159ce44dfd"}
Jul 26 15:42:16.551974 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.551785    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md" event={"ID":"08469263-faa4-4324-893f-a9d6e79658cf","Type":"ContainerStarted","Data":"00db919c4ba4b72482d9b1ba20c9362b54bf8a009c7d5beaf65bc7b30a46631f"}
Jul 26 15:42:16.553519 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.553489    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv" event={"ID":"8dee3d7e-c67e-484e-97ea-67917717ecee","Type":"ContainerStarted","Data":"ac5747dd7035f80b8c8c0ace8140955942a37da164b0eb5dcfe8420ede524c4f"}
Jul 26 15:42:16.553652 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.553552    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:16.569211 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.569165    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4j9md" podStartSLOduration=2.569151837 podStartE2EDuration="2.569151837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:42:16.568019369 +0000 UTC m=+235.752085507" watchObservedRunningTime="2026-07-26 15:42:16.569151837 +0000 UTC m=+235.753217975"
Jul 26 15:42:16.585703 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:16.585639    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv" podStartSLOduration=1.807696999 podStartE2EDuration="5.585618322s" totalImagesPullingTime="3.777921323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:42:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:42:16.583831614 +0000 UTC m=+235.767897753" watchObservedRunningTime="2026-07-26 15:42:16.585618322 +0000 UTC m=+235.769684459"
Jul 26 15:42:22.559684 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:22.559655    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-crvfv"
Jul 26 15:42:44.386324 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.386246    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"]
Jul 26 15:42:44.389668 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.389652    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.392085 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.392058    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 15:42:44.392200 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.392164    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 15:42:44.392280 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.392215    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tgn28\""
Jul 26 15:42:44.392368 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.392350    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 15:42:44.392510 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.392498    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 15:42:44.407209 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.407187    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"]
Jul 26 15:42:44.494515 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.494489    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrf6h\" (UniqueName: \"kubernetes.io/projected/df18ad45-2d65-4175-9647-e4a654d79132-kube-api-access-hrf6h\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.494643 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.494533    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df18ad45-2d65-4175-9647-e4a654d79132-manifests\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.494643 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.494550    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df18ad45-2d65-4175-9647-e4a654d79132-tmp\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.494643 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.494628    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df18ad45-2d65-4175-9647-e4a654d79132-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.494748 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.494678    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df18ad45-2d65-4175-9647-e4a654d79132-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.595785 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.595755    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df18ad45-2d65-4175-9647-e4a654d79132-manifests\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.595902 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.595789    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df18ad45-2d65-4175-9647-e4a654d79132-tmp\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.595902 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.595823    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df18ad45-2d65-4175-9647-e4a654d79132-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.595902 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.595855    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df18ad45-2d65-4175-9647-e4a654d79132-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.596014 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.595906    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrf6h\" (UniqueName: \"kubernetes.io/projected/df18ad45-2d65-4175-9647-e4a654d79132-kube-api-access-hrf6h\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.596258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.596198    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/df18ad45-2d65-4175-9647-e4a654d79132-manifests\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.598191 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.598158    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df18ad45-2d65-4175-9647-e4a654d79132-tmp\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.598373 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.598350    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/df18ad45-2d65-4175-9647-e4a654d79132-webhook-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.598417 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.598356    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/df18ad45-2d65-4175-9647-e4a654d79132-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.603985 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.603961    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrf6h\" (UniqueName: \"kubernetes.io/projected/df18ad45-2d65-4175-9647-e4a654d79132-kube-api-access-hrf6h\") pod \"opendatahub-operator-controller-manager-66d4685587-8nzs5\" (UID: \"df18ad45-2d65-4175-9647-e4a654d79132\") " pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.699042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.699019    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:44.837379 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:44.837357    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"]
Jul 26 15:42:44.840272 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:42:44.840219    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf18ad45_2d65_4175_9647_e4a654d79132.slice/crio-b92f40d641f7112e465212aefec2b8d64197fe878d5e56090fed979ae6e7bacb WatchSource:0}: Error finding container b92f40d641f7112e465212aefec2b8d64197fe878d5e56090fed979ae6e7bacb: Status 404 returned error can't find the container with id b92f40d641f7112e465212aefec2b8d64197fe878d5e56090fed979ae6e7bacb
Jul 26 15:42:45.677393 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:45.677341    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5" event={"ID":"df18ad45-2d65-4175-9647-e4a654d79132","Type":"ContainerStarted","Data":"b92f40d641f7112e465212aefec2b8d64197fe878d5e56090fed979ae6e7bacb"}
Jul 26 15:42:48.693938 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:48.693902    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5" event={"ID":"df18ad45-2d65-4175-9647-e4a654d79132","Type":"ContainerStarted","Data":"9920c104e17a05f7b1fd49a5ae22b54ae16f586a8deac8532c1d71cbf6145a5f"}
Jul 26 15:42:49.700505 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:49.700471    2534 generic.go:356] "Generic (PLEG): container finished" podID="df18ad45-2d65-4175-9647-e4a654d79132" containerID="9920c104e17a05f7b1fd49a5ae22b54ae16f586a8deac8532c1d71cbf6145a5f" exitCode=0
Jul 26 15:42:49.700900 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:49.700568    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5" event={"ID":"df18ad45-2d65-4175-9647-e4a654d79132","Type":"ContainerDied","Data":"9920c104e17a05f7b1fd49a5ae22b54ae16f586a8deac8532c1d71cbf6145a5f"}
Jul 26 15:42:50.706441 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:50.706410    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5" event={"ID":"df18ad45-2d65-4175-9647-e4a654d79132","Type":"ContainerStarted","Data":"d2c3ca7cdad0c4b261517c0da7a8f2a2a7fe300cc8a52e5f7f869736641988f3"}
Jul 26 15:42:50.706829 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:50.706525    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:42:50.726868 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:42:50.726819    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5" podStartSLOduration=2.42459372 podStartE2EDuration="6.72680052s" totalImagesPullingTime="3.281320768s" totalInitContainerRuntime="1.020886032s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:42:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:42:50.725022561 +0000 UTC m=+269.909088700" watchObservedRunningTime="2026-07-26 15:42:50.72680052 +0000 UTC m=+269.910866658"
Jul 26 15:43:01.712946 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:01.712915    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66d4685587-8nzs5"
Jul 26 15:43:04.351740 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.351701    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"]
Jul 26 15:43:04.360095 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.360056    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.362648 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.362625    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 26 15:43:04.363653 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.363619    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 26 15:43:04.363653 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.363645    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hqcjz\""
Jul 26 15:43:04.363821 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.363649    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 26 15:43:04.363821 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.363751    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 26 15:43:04.363821 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.363655    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 26 15:43:04.366200 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.366177    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"]
Jul 26 15:43:04.467794 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.467764    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6b3291c-0027-488b-b524-4ec99718160d-cert\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.467974 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.467807    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6b3291c-0027-488b-b524-4ec99718160d-metrics-cert\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.467974 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.467905    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6b3291c-0027-488b-b524-4ec99718160d-manager-config\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.467974 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.467960    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz88g\" (UniqueName: \"kubernetes.io/projected/f6b3291c-0027-488b-b524-4ec99718160d-kube-api-access-dz88g\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.568643 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.568606    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6b3291c-0027-488b-b524-4ec99718160d-manager-config\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.568777 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.568664    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz88g\" (UniqueName: \"kubernetes.io/projected/f6b3291c-0027-488b-b524-4ec99718160d-kube-api-access-dz88g\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.568777 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.568714    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6b3291c-0027-488b-b524-4ec99718160d-cert\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.568777 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.568744    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6b3291c-0027-488b-b524-4ec99718160d-metrics-cert\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.569427 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.569402    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6b3291c-0027-488b-b524-4ec99718160d-manager-config\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.571131 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.571108    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6b3291c-0027-488b-b524-4ec99718160d-cert\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.571265 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.571183    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6b3291c-0027-488b-b524-4ec99718160d-metrics-cert\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.580211 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.580184    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz88g\" (UniqueName: \"kubernetes.io/projected/f6b3291c-0027-488b-b524-4ec99718160d-kube-api-access-dz88g\") pod \"lws-controller-manager-54c98bf85c-q226v\" (UID: \"f6b3291c-0027-488b-b524-4ec99718160d\") " pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.671119 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.671031    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:04.811858 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:04.811826    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"]
Jul 26 15:43:04.815285 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:43:04.815254    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b3291c_0027_488b_b524_4ec99718160d.slice/crio-59b4378cee88ffef69fe192e5102b69e5a5f06e72ec193db15ce9e0035d4bb30 WatchSource:0}: Error finding container 59b4378cee88ffef69fe192e5102b69e5a5f06e72ec193db15ce9e0035d4bb30: Status 404 returned error can't find the container with id 59b4378cee88ffef69fe192e5102b69e5a5f06e72ec193db15ce9e0035d4bb30
Jul 26 15:43:05.774151 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:05.774117    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v" event={"ID":"f6b3291c-0027-488b-b524-4ec99718160d","Type":"ContainerStarted","Data":"59b4378cee88ffef69fe192e5102b69e5a5f06e72ec193db15ce9e0035d4bb30"}
Jul 26 15:43:09.793411 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:09.793374    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v" event={"ID":"f6b3291c-0027-488b-b524-4ec99718160d","Type":"ContainerStarted","Data":"4198f08a2ec59f3da0701a9aaae2a5c11eb35b7d69cc0516e03962827bc32b3b"}
Jul 26 15:43:09.793807 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:09.793531    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:09.813901 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:09.813796    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v" podStartSLOduration=1.077199061 podStartE2EDuration="5.813780048s" totalImagesPullingTime="4.736580987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:43:09.812868449 +0000 UTC m=+288.996934588" watchObservedRunningTime="2026-07-26 15:43:09.813780048 +0000 UTC m=+288.997846185"
Jul 26 15:43:14.081922 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.081887    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"]
Jul 26 15:43:14.085135 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.085116    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.088381 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.088360    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 15:43:14.090957 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.090937    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 15:43:14.091048 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.090942    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-n48rr\""
Jul 26 15:43:14.091107 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.090943    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 15:43:14.091163 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.090947    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 15:43:14.113855 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.113825    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"]
Jul 26 15:43:14.159081 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.159049    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-tmp\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.159225 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.159097    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrnk5\" (UniqueName: \"kubernetes.io/projected/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-kube-api-access-zrnk5\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.159225 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.159144    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-tls-certs\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.260064 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.260029    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-tmp\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.260216 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.260080    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrnk5\" (UniqueName: \"kubernetes.io/projected/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-kube-api-access-zrnk5\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.260216 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.260107    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-tls-certs\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.262310 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.262288    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-tmp\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.262520 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.262503    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-tls-certs\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.270303 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.270280    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrnk5\" (UniqueName: \"kubernetes.io/projected/c09f38e6-ee41-4c16-b61f-c4012f6a78ac-kube-api-access-zrnk5\") pod \"kube-auth-proxy-694bf5b6d6-mvv4m\" (UID: \"c09f38e6-ee41-4c16-b61f-c4012f6a78ac\") " pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.394068 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.393996    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"
Jul 26 15:43:14.561939 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.561912    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m"]
Jul 26 15:43:14.564745 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:43:14.564712    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc09f38e6_ee41_4c16_b61f_c4012f6a78ac.slice/crio-49ca591ed3ba3b442b6e7d9ecc6c174e4af2f81b449377bb4c739e172c940ab6 WatchSource:0}: Error finding container 49ca591ed3ba3b442b6e7d9ecc6c174e4af2f81b449377bb4c739e172c940ab6: Status 404 returned error can't find the container with id 49ca591ed3ba3b442b6e7d9ecc6c174e4af2f81b449377bb4c739e172c940ab6
Jul 26 15:43:14.820435 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:14.820401    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m" event={"ID":"c09f38e6-ee41-4c16-b61f-c4012f6a78ac","Type":"ContainerStarted","Data":"49ca591ed3ba3b442b6e7d9ecc6c174e4af2f81b449377bb4c739e172c940ab6"}
Jul 26 15:43:17.835578 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:17.835542    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m" event={"ID":"c09f38e6-ee41-4c16-b61f-c4012f6a78ac","Type":"ContainerStarted","Data":"55d6afbfd40b353373400252e18a23c568e982072223799909350b2b9d580ae7"}
Jul 26 15:43:17.857690 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:17.857634    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-694bf5b6d6-mvv4m" podStartSLOduration=1.13516753 podStartE2EDuration="3.857619362s" totalImagesPullingTime="2.722451832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:43:17.856034256 +0000 UTC m=+297.040100399" watchObservedRunningTime="2026-07-26 15:43:17.857619362 +0000 UTC m=+297.041685498"
Jul 26 15:43:20.799736 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:20.799706    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54c98bf85c-q226v"
Jul 26 15:43:21.141446 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:43:21.141370    2534 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 15:44:48.815414 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.815377    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bb45cc56f-nxg6q"]
Jul 26 15:44:48.818761 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.818742    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.821268 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.821244    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 26 15:44:48.821402 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.821386    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 26 15:44:48.821452 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.821431    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 26 15:44:48.821497 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.821471    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 26 15:44:48.822311 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.822294    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 26 15:44:48.822409 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.822316    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 26 15:44:48.822495 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.822480    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 26 15:44:48.822647 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.822630    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-rr5w4\""
Jul 26 15:44:48.827207 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.827188    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 26 15:44:48.833651 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.833630    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb45cc56f-nxg6q"]
Jul 26 15:44:48.988544 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988508    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-service-ca\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.988709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988574    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-trusted-ca-bundle\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.988709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988602    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-oauth-config\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.988709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988623    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-serving-cert\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.988709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988676    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-config\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.988709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988705    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l8qv\" (UniqueName: \"kubernetes.io/projected/c697675b-acd7-4a74-ac80-1c0c84346e7f-kube-api-access-8l8qv\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:48.988872 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:48.988733    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-oauth-serving-cert\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089404 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089326    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-trusted-ca-bundle\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089404 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089365    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-oauth-config\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089404 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089395    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-serving-cert\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089663 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089415    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-config\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089663 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089437    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l8qv\" (UniqueName: \"kubernetes.io/projected/c697675b-acd7-4a74-ac80-1c0c84346e7f-kube-api-access-8l8qv\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089663 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089457    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-oauth-serving-cert\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.089663 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.089604    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-service-ca\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.090129 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.090101    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-config\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.090295 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.090275    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-trusted-ca-bundle\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.090354 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.090315    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-service-ca\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.090395 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.090359    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c697675b-acd7-4a74-ac80-1c0c84346e7f-oauth-serving-cert\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.091916 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.091894    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-oauth-config\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.092090 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.092072    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c697675b-acd7-4a74-ac80-1c0c84346e7f-console-serving-cert\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.097910 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.097891    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l8qv\" (UniqueName: \"kubernetes.io/projected/c697675b-acd7-4a74-ac80-1c0c84346e7f-kube-api-access-8l8qv\") pod \"console-6bb45cc56f-nxg6q\" (UID: \"c697675b-acd7-4a74-ac80-1c0c84346e7f\") " pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.128925 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.128903    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:49.255973 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:49.255947    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb45cc56f-nxg6q"]
Jul 26 15:44:49.258537 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:44:49.258512    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc697675b_acd7_4a74_ac80_1c0c84346e7f.slice/crio-c622dab5137c12a1094b2322ff6c5524d1228f49afba81206b1e3158af5673c3 WatchSource:0}: Error finding container c622dab5137c12a1094b2322ff6c5524d1228f49afba81206b1e3158af5673c3: Status 404 returned error can't find the container with id c622dab5137c12a1094b2322ff6c5524d1228f49afba81206b1e3158af5673c3
Jul 26 15:44:50.261002 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:50.260963    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb45cc56f-nxg6q" event={"ID":"c697675b-acd7-4a74-ac80-1c0c84346e7f","Type":"ContainerStarted","Data":"88ebcd8bd369f4f6d9ff474b68d7ced189c8d962ccfcfc2bcce306eb531525d4"}
Jul 26 15:44:50.261437 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:50.261021    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb45cc56f-nxg6q" event={"ID":"c697675b-acd7-4a74-ac80-1c0c84346e7f","Type":"ContainerStarted","Data":"c622dab5137c12a1094b2322ff6c5524d1228f49afba81206b1e3158af5673c3"}
Jul 26 15:44:50.285807 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:50.280829    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bb45cc56f-nxg6q" podStartSLOduration=2.280809375 podStartE2EDuration="2.280809375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:44:50.278383597 +0000 UTC m=+389.462449734" watchObservedRunningTime="2026-07-26 15:44:50.280809375 +0000 UTC m=+389.464875514"
Jul 26 15:44:55.784851 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.784813    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"]
Jul 26 15:44:55.788191 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.788171    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:55.790638 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.790618    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 26 15:44:55.790741 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.790641    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 26 15:44:55.790803 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.790735    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sjb8w\""
Jul 26 15:44:55.790803 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.790784    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 15:44:55.791537 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.791516    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 15:44:55.797721 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.797703    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"]
Jul 26 15:44:55.952725 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.952692    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/44c30aa3-d742-4a75-8cb9-b80cc92788d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:55.952903 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.952748    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/44c30aa3-d742-4a75-8cb9-b80cc92788d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:55.952903 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:55.952808    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5znm\" (UniqueName: \"kubernetes.io/projected/44c30aa3-d742-4a75-8cb9-b80cc92788d3-kube-api-access-b5znm\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.054354 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.054257    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/44c30aa3-d742-4a75-8cb9-b80cc92788d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.054354 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.054300    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5znm\" (UniqueName: \"kubernetes.io/projected/44c30aa3-d742-4a75-8cb9-b80cc92788d3-kube-api-access-b5znm\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.054575 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.054393    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/44c30aa3-d742-4a75-8cb9-b80cc92788d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.054955 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.054929    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/44c30aa3-d742-4a75-8cb9-b80cc92788d3-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.056706 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.056679    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/44c30aa3-d742-4a75-8cb9-b80cc92788d3-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.063676 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.063652    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5znm\" (UniqueName: \"kubernetes.io/projected/44c30aa3-d742-4a75-8cb9-b80cc92788d3-kube-api-access-b5znm\") pod \"kuadrant-console-plugin-66b5774985-29gdb\" (UID: \"44c30aa3-d742-4a75-8cb9-b80cc92788d3\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.104675 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.104644    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"
Jul 26 15:44:56.228338 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.228290    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb"]
Jul 26 15:44:56.230482 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:44:56.230455    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44c30aa3_d742_4a75_8cb9_b80cc92788d3.slice/crio-563e0e7da7bef5e85f6f13dc8885f7444b07f480d265c54f5c41e829104c02d3 WatchSource:0}: Error finding container 563e0e7da7bef5e85f6f13dc8885f7444b07f480d265c54f5c41e829104c02d3: Status 404 returned error can't find the container with id 563e0e7da7bef5e85f6f13dc8885f7444b07f480d265c54f5c41e829104c02d3
Jul 26 15:44:56.231781 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.231761    2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 15:44:56.292258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:56.292218    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb" event={"ID":"44c30aa3-d742-4a75-8cb9-b80cc92788d3","Type":"ContainerStarted","Data":"563e0e7da7bef5e85f6f13dc8885f7444b07f480d265c54f5c41e829104c02d3"}
Jul 26 15:44:59.130041 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:59.130006    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:59.130531 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:59.130344    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:59.135441 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:59.135417    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:44:59.312340 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:44:59.312272    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bb45cc56f-nxg6q"
Jul 26 15:45:06.009077 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.009037    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"]
Jul 26 15:45:06.042893 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.042863    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"]
Jul 26 15:45:06.043069 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.042980    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.045815 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.045790    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7fjh9\""
Jul 26 15:45:06.055020 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.054995    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqqfq\" (UniqueName: \"kubernetes.io/projected/304c9bd2-9c66-47dd-880c-413233f84699-kube-api-access-rqqfq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.055150 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.055078    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/304c9bd2-9c66-47dd-880c-413233f84699-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.156181 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.156130    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqqfq\" (UniqueName: \"kubernetes.io/projected/304c9bd2-9c66-47dd-880c-413233f84699-kube-api-access-rqqfq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.156381 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.156260    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/304c9bd2-9c66-47dd-880c-413233f84699-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.156634 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.156613    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/304c9bd2-9c66-47dd-880c-413233f84699-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.166415 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.166388    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqqfq\" (UniqueName: \"kubernetes.io/projected/304c9bd2-9c66-47dd-880c-413233f84699-kube-api-access-rqqfq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.355423 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.355351    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:06.640980 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.640907    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"]
Jul 26 15:45:06.652014 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.651977    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"]
Jul 26 15:45:06.664595 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.664555    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"]
Jul 26 15:45:06.680488 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.680464    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.688678 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.688640    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"]
Jul 26 15:45:06.760881 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.760851    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmw6\" (UniqueName: \"kubernetes.io/projected/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-kube-api-access-sxmw6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cnl2n\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.761026 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.760899    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cnl2n\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.861824 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.861788    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmw6\" (UniqueName: \"kubernetes.io/projected/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-kube-api-access-sxmw6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cnl2n\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.862019 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.861847    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cnl2n\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.862224 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.862203    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cnl2n\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.870537 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.870505    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmw6\" (UniqueName: \"kubernetes.io/projected/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-kube-api-access-sxmw6\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-cnl2n\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:06.994818 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:06.994783    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:16.869904 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:16.869875    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"]
Jul 26 15:45:16.871992 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:45:16.871968    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c0681ce_b2ca_4361_9e38_3d20ab97b6b3.slice/crio-8f8d16d3bd817262762f011dd7bf8f0bf55534c962640310c29dee61fcf74049 WatchSource:0}: Error finding container 8f8d16d3bd817262762f011dd7bf8f0bf55534c962640310c29dee61fcf74049: Status 404 returned error can't find the container with id 8f8d16d3bd817262762f011dd7bf8f0bf55534c962640310c29dee61fcf74049
Jul 26 15:45:17.407569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:17.407525    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n" event={"ID":"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3","Type":"ContainerStarted","Data":"8f8d16d3bd817262762f011dd7bf8f0bf55534c962640310c29dee61fcf74049"}
Jul 26 15:45:17.409133 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:17.409108    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb" event={"ID":"44c30aa3-d742-4a75-8cb9-b80cc92788d3","Type":"ContainerStarted","Data":"ed5ff30be16ac4acfccb7be445175e046390560c5fd9573732247380a4137934"}
Jul 26 15:45:17.425793 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:17.425729    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-29gdb" podStartSLOduration=1.840900785 podStartE2EDuration="22.425708374s" totalImagesPullingTime="20.584807589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:45:17.423752046 +0000 UTC m=+416.607818187" watchObservedRunningTime="2026-07-26 15:45:17.425708374 +0000 UTC m=+416.609774512"
Jul 26 15:45:19.271625 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:19.271585    2534 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-fxwmd_kuadrant-system_304c9bd2-9c66-47dd-880c-413233f84699_0(3ef36f20c3306b0b388e4f0a3703501505e0113703cef555f05b0e169b09b242): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-fxwmd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 26 15:45:19.271878 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:19.271652    2534 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-fxwmd_kuadrant-system_304c9bd2-9c66-47dd-880c-413233f84699_0(3ef36f20c3306b0b388e4f0a3703501505e0113703cef555f05b0e169b09b242): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-fxwmd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:19.420047 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.420012    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:19.422379 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:19.422338    2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" is forbidden: User \"system:node:ip-10-0-138-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-109.ec2.internal' and this object" podUID="304c9bd2-9c66-47dd-880c-413233f84699" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:19.428818 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.428795    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:19.430673 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:19.430649    2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" is forbidden: User \"system:node:ip-10-0-138-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-109.ec2.internal' and this object" podUID="304c9bd2-9c66-47dd-880c-413233f84699" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:19.488287 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.488255    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqqfq\" (UniqueName: \"kubernetes.io/projected/304c9bd2-9c66-47dd-880c-413233f84699-kube-api-access-rqqfq\") pod \"304c9bd2-9c66-47dd-880c-413233f84699\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") "
Jul 26 15:45:19.488445 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.488385    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/304c9bd2-9c66-47dd-880c-413233f84699-extensions-socket-volume\") pod \"304c9bd2-9c66-47dd-880c-413233f84699\" (UID: \"304c9bd2-9c66-47dd-880c-413233f84699\") "
Jul 26 15:45:19.488718 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.488686    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/304c9bd2-9c66-47dd-880c-413233f84699-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "304c9bd2-9c66-47dd-880c-413233f84699" (UID: "304c9bd2-9c66-47dd-880c-413233f84699"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:45:19.490512 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.490486    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/304c9bd2-9c66-47dd-880c-413233f84699-kube-api-access-rqqfq" (OuterVolumeSpecName: "kube-api-access-rqqfq") pod "304c9bd2-9c66-47dd-880c-413233f84699" (UID: "304c9bd2-9c66-47dd-880c-413233f84699"). InnerVolumeSpecName "kube-api-access-rqqfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:45:19.590068 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.589994    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqqfq\" (UniqueName: \"kubernetes.io/projected/304c9bd2-9c66-47dd-880c-413233f84699-kube-api-access-rqqfq\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:45:19.590068 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:19.590030    2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/304c9bd2-9c66-47dd-880c-413233f84699-extensions-socket-volume\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:45:20.426091 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:20.425981    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:20.428556 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:20.428522    2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" is forbidden: User \"system:node:ip-10-0-138-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-109.ec2.internal' and this object" podUID="304c9bd2-9c66-47dd-880c-413233f84699" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:20.436308 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:20.436276    2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" is forbidden: User \"system:node:ip-10-0-138-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-109.ec2.internal' and this object" podUID="304c9bd2-9c66-47dd-880c-413233f84699" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:21.292435 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:21.292391    2534 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-fxwmd\" is forbidden: User \"system:node:ip-10-0-138-109.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-109.ec2.internal' and this object" podUID="304c9bd2-9c66-47dd-880c-413233f84699" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-fxwmd"
Jul 26 15:45:21.295125 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:21.295099    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="304c9bd2-9c66-47dd-880c-413233f84699" path="/var/lib/kubelet/pods/304c9bd2-9c66-47dd-880c-413233f84699/volumes"
Jul 26 15:45:22.437037 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:22.436991    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n" event={"ID":"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3","Type":"ContainerStarted","Data":"c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d"}
Jul 26 15:45:22.437499 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:22.437203    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:22.456709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:22.456644    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n" podStartSLOduration=11.769762308 podStartE2EDuration="16.456626381s" totalImagesPullingTime="4.686864073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:45:22.455501757 +0000 UTC m=+421.639567896" watchObservedRunningTime="2026-07-26 15:45:22.456626381 +0000 UTC m=+421.640692528"
Jul 26 15:45:33.444050 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:33.444017    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:46.253342 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.253270    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"]
Jul 26 15:45:46.253845 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.253548    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n" podUID="8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" containerName="manager" containerID="cri-o://c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d" gracePeriod=10
Jul 26 15:45:46.508413 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.508357    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:46.525368 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.525343    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-extensions-socket-volume\") pod \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") "
Jul 26 15:45:46.525477 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.525435    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxmw6\" (UniqueName: \"kubernetes.io/projected/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-kube-api-access-sxmw6\") pod \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\" (UID: \"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3\") "
Jul 26 15:45:46.525777 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.525744    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" (UID: "8c0681ce-b2ca-4361-9e38-3d20ab97b6b3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 15:45:46.528042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.528018    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-kube-api-access-sxmw6" (OuterVolumeSpecName: "kube-api-access-sxmw6") pod "8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" (UID: "8c0681ce-b2ca-4361-9e38-3d20ab97b6b3"). InnerVolumeSpecName "kube-api-access-sxmw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:45:46.544627 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.544598    2534 generic.go:356] "Generic (PLEG): container finished" podID="8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" containerID="c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d" exitCode=0
Jul 26 15:45:46.544723 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.544642    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"
Jul 26 15:45:46.544723 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.544675    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n" event={"ID":"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3","Type":"ContainerDied","Data":"c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d"}
Jul 26 15:45:46.544810 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.544734    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n" event={"ID":"8c0681ce-b2ca-4361-9e38-3d20ab97b6b3","Type":"ContainerDied","Data":"8f8d16d3bd817262762f011dd7bf8f0bf55534c962640310c29dee61fcf74049"}
Jul 26 15:45:46.544810 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.544764    2534 scope.go:117] "RemoveContainer" containerID="c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d"
Jul 26 15:45:46.553558 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.553540    2534 scope.go:117] "RemoveContainer" containerID="c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d"
Jul 26 15:45:46.553851 ip-10-0-138-109 kubenswrapper[2534]: E0726 15:45:46.553832    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d\": container with ID starting with c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d not found: ID does not exist" containerID="c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d"
Jul 26 15:45:46.553907 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.553859    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d"} err="failed to get container status \"c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d\": rpc error: code = NotFound desc = could not find container \"c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d\": container with ID starting with c232b4b1b3717469a3125974c535d8d0245b8d897225ba35f58084bd91b4b23d not found: ID does not exist"
Jul 26 15:45:46.565702 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.565682    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"]
Jul 26 15:45:46.573106 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.573087    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-cnl2n"]
Jul 26 15:45:46.626539 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.626510    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxmw6\" (UniqueName: \"kubernetes.io/projected/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-kube-api-access-sxmw6\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:45:46.626539 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:46.626532    2534 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3-extensions-socket-volume\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:45:47.294375 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:47.294337    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" path="/var/lib/kubelet/pods/8c0681ce-b2ca-4361-9e38-3d20ab97b6b3/volumes"
Jul 26 15:45:53.295434 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.295405    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:45:53.295879 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.295747    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" containerName="manager"
Jul 26 15:45:53.295879 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.295757    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" containerName="manager"
Jul 26 15:45:53.295879 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.295819    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c0681ce-b2ca-4361-9e38-3d20ab97b6b3" containerName="manager"
Jul 26 15:45:53.329126 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.329098    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:45:53.329126 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.329129    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:45:53.329323 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.329209    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.331595 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.331575    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 15:45:53.474619 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.474581    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flgsl\" (UniqueName: \"kubernetes.io/projected/920b7045-26d5-4c44-a1f6-625ee4fc2982-kube-api-access-flgsl\") pod \"limitador-limitador-54d6d988d5-jhgzg\" (UID: \"920b7045-26d5-4c44-a1f6-625ee4fc2982\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.474803 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.474781    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/920b7045-26d5-4c44-a1f6-625ee4fc2982-config-file\") pod \"limitador-limitador-54d6d988d5-jhgzg\" (UID: \"920b7045-26d5-4c44-a1f6-625ee4fc2982\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.575510 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.575434    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/920b7045-26d5-4c44-a1f6-625ee4fc2982-config-file\") pod \"limitador-limitador-54d6d988d5-jhgzg\" (UID: \"920b7045-26d5-4c44-a1f6-625ee4fc2982\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.575646 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.575527    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flgsl\" (UniqueName: \"kubernetes.io/projected/920b7045-26d5-4c44-a1f6-625ee4fc2982-kube-api-access-flgsl\") pod \"limitador-limitador-54d6d988d5-jhgzg\" (UID: \"920b7045-26d5-4c44-a1f6-625ee4fc2982\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.576014 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.575994    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/920b7045-26d5-4c44-a1f6-625ee4fc2982-config-file\") pod \"limitador-limitador-54d6d988d5-jhgzg\" (UID: \"920b7045-26d5-4c44-a1f6-625ee4fc2982\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.582811 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.582786    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flgsl\" (UniqueName: \"kubernetes.io/projected/920b7045-26d5-4c44-a1f6-625ee4fc2982-kube-api-access-flgsl\") pod \"limitador-limitador-54d6d988d5-jhgzg\" (UID: \"920b7045-26d5-4c44-a1f6-625ee4fc2982\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.639205 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.639181    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:53.791896 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.791862    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:45:53.795718 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:45:53.795681    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920b7045_26d5_4c44_a1f6_625ee4fc2982.slice/crio-5d831edc9cf6ca6ac5cf79366ecf16b577d13388c9dfe1a8434d0b5c197cb1f8 WatchSource:0}: Error finding container 5d831edc9cf6ca6ac5cf79366ecf16b577d13388c9dfe1a8434d0b5c197cb1f8: Status 404 returned error can't find the container with id 5d831edc9cf6ca6ac5cf79366ecf16b577d13388c9dfe1a8434d0b5c197cb1f8
Jul 26 15:45:53.867779 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.867717    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzpjd"]
Jul 26 15:45:53.897010 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.896982    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzpjd"]
Jul 26 15:45:53.897136 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.897019    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:45:53.899365 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.899347    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-496wt\""
Jul 26 15:45:53.979007 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:53.978978    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7m96\" (UniqueName: \"kubernetes.io/projected/e2e5d348-1e4b-41fd-b85e-99d56d41a37d-kube-api-access-s7m96\") pod \"authorino-75d556bd77-pzpjd\" (UID: \"e2e5d348-1e4b-41fd-b85e-99d56d41a37d\") " pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:45:54.079977 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:54.079942    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7m96\" (UniqueName: \"kubernetes.io/projected/e2e5d348-1e4b-41fd-b85e-99d56d41a37d-kube-api-access-s7m96\") pod \"authorino-75d556bd77-pzpjd\" (UID: \"e2e5d348-1e4b-41fd-b85e-99d56d41a37d\") " pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:45:54.087939 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:54.087918    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7m96\" (UniqueName: \"kubernetes.io/projected/e2e5d348-1e4b-41fd-b85e-99d56d41a37d-kube-api-access-s7m96\") pod \"authorino-75d556bd77-pzpjd\" (UID: \"e2e5d348-1e4b-41fd-b85e-99d56d41a37d\") " pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:45:54.205935 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:54.205904    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:45:54.356703 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:54.356674    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzpjd"]
Jul 26 15:45:54.358624 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:45:54.358595    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2e5d348_1e4b_41fd_b85e_99d56d41a37d.slice/crio-56fc4fe98b77a1a8cadcc9dce6ea6255438f03cb2ed60958b6786e1591fd5bf4 WatchSource:0}: Error finding container 56fc4fe98b77a1a8cadcc9dce6ea6255438f03cb2ed60958b6786e1591fd5bf4: Status 404 returned error can't find the container with id 56fc4fe98b77a1a8cadcc9dce6ea6255438f03cb2ed60958b6786e1591fd5bf4
Jul 26 15:45:54.585884 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:54.585768    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzpjd" event={"ID":"e2e5d348-1e4b-41fd-b85e-99d56d41a37d","Type":"ContainerStarted","Data":"56fc4fe98b77a1a8cadcc9dce6ea6255438f03cb2ed60958b6786e1591fd5bf4"}
Jul 26 15:45:54.588046 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:54.587990    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg" event={"ID":"920b7045-26d5-4c44-a1f6-625ee4fc2982","Type":"ContainerStarted","Data":"5d831edc9cf6ca6ac5cf79366ecf16b577d13388c9dfe1a8434d0b5c197cb1f8"}
Jul 26 15:45:58.612900 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:58.612804    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzpjd" event={"ID":"e2e5d348-1e4b-41fd-b85e-99d56d41a37d","Type":"ContainerStarted","Data":"98af1f262a145f986d4a7f22b6310a53efd537ad508a99f12f03bbf4ae900e61"}
Jul 26 15:45:58.614292 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:58.614267    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg" event={"ID":"920b7045-26d5-4c44-a1f6-625ee4fc2982","Type":"ContainerStarted","Data":"6cb90201785c4cb48b20f28f88f170e0bf95d42417e5471f0aed566b52e5bec9"}
Jul 26 15:45:58.614412 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:58.614394    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:45:58.627198 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:58.627151    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-pzpjd" podStartSLOduration=1.6341606899999999 podStartE2EDuration="5.627136653s" totalImagesPullingTime="3.992975963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:45:58.626262989 +0000 UTC m=+457.810329127" watchObservedRunningTime="2026-07-26 15:45:58.627136653 +0000 UTC m=+457.811202789"
Jul 26 15:45:58.641679 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:45:58.641616    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg" podStartSLOduration=1.093981658 podStartE2EDuration="5.641593738s" totalImagesPullingTime="4.54761208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:45:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:45:58.640639053 +0000 UTC m=+457.824705191" watchObservedRunningTime="2026-07-26 15:45:58.641593738 +0000 UTC m=+457.825659876"
Jul 26 15:46:09.620360 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:09.620329    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"
Jul 26 15:46:28.545597 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.545563    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzpjd"]
Jul 26 15:46:28.546042 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.545762    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-pzpjd" podUID="e2e5d348-1e4b-41fd-b85e-99d56d41a37d" containerName="authorino" containerID="cri-o://98af1f262a145f986d4a7f22b6310a53efd537ad508a99f12f03bbf4ae900e61" gracePeriod=30
Jul 26 15:46:28.751767 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.751738    2534 generic.go:356] "Generic (PLEG): container finished" podID="e2e5d348-1e4b-41fd-b85e-99d56d41a37d" containerID="98af1f262a145f986d4a7f22b6310a53efd537ad508a99f12f03bbf4ae900e61" exitCode=0
Jul 26 15:46:28.751910 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.751807    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzpjd" event={"ID":"e2e5d348-1e4b-41fd-b85e-99d56d41a37d","Type":"ContainerDied","Data":"98af1f262a145f986d4a7f22b6310a53efd537ad508a99f12f03bbf4ae900e61"}
Jul 26 15:46:28.792097 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.792076    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:46:28.884814 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.884757    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7m96\" (UniqueName: \"kubernetes.io/projected/e2e5d348-1e4b-41fd-b85e-99d56d41a37d-kube-api-access-s7m96\") pod \"e2e5d348-1e4b-41fd-b85e-99d56d41a37d\" (UID: \"e2e5d348-1e4b-41fd-b85e-99d56d41a37d\") "
Jul 26 15:46:28.886733 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.886708    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2e5d348-1e4b-41fd-b85e-99d56d41a37d-kube-api-access-s7m96" (OuterVolumeSpecName: "kube-api-access-s7m96") pod "e2e5d348-1e4b-41fd-b85e-99d56d41a37d" (UID: "e2e5d348-1e4b-41fd-b85e-99d56d41a37d"). InnerVolumeSpecName "kube-api-access-s7m96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 15:46:28.986381 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:28.986346    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7m96\" (UniqueName: \"kubernetes.io/projected/e2e5d348-1e4b-41fd-b85e-99d56d41a37d-kube-api-access-s7m96\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 15:46:29.757168 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:29.757140    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzpjd"
Jul 26 15:46:29.757588 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:29.757139    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzpjd" event={"ID":"e2e5d348-1e4b-41fd-b85e-99d56d41a37d","Type":"ContainerDied","Data":"56fc4fe98b77a1a8cadcc9dce6ea6255438f03cb2ed60958b6786e1591fd5bf4"}
Jul 26 15:46:29.757588 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:29.757285    2534 scope.go:117] "RemoveContainer" containerID="98af1f262a145f986d4a7f22b6310a53efd537ad508a99f12f03bbf4ae900e61"
Jul 26 15:46:29.772284 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:29.772260    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzpjd"]
Jul 26 15:46:29.776242 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:29.776209    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzpjd"]
Jul 26 15:46:31.293760 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:46:31.293725    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2e5d348-1e4b-41fd-b85e-99d56d41a37d" path="/var/lib/kubelet/pods/e2e5d348-1e4b-41fd-b85e-99d56d41a37d/volumes"
Jul 26 15:47:10.987203 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:10.987171    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:47:28.609872 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:28.609836    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:47:35.551561 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.551478    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"]
Jul 26 15:47:35.552091 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.552041    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2e5d348-1e4b-41fd-b85e-99d56d41a37d" containerName="authorino"
Jul 26 15:47:35.552091 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.552061    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2e5d348-1e4b-41fd-b85e-99d56d41a37d" containerName="authorino"
Jul 26 15:47:35.552258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.552159    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2e5d348-1e4b-41fd-b85e-99d56d41a37d" containerName="authorino"
Jul 26 15:47:35.557116 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.557095    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.559541 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.559521    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 15:47:35.560703 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.560430    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 26 15:47:35.560703 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.560451    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 15:47:35.560703 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.560456    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6w9zr\""
Jul 26 15:47:35.563599 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.563577    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"]
Jul 26 15:47:35.575420 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.575393    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.575530 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.575436    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqbr6\" (UniqueName: \"kubernetes.io/projected/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-kube-api-access-gqbr6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.575597 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.575556    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.575652 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.575627    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.575709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.575677    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.575709 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.575703    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.676604 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.676571    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.676784 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.676614    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.676784 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.676635    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.676784 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.676675    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.676784 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.676691    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqbr6\" (UniqueName: \"kubernetes.io/projected/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-kube-api-access-gqbr6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.676784 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.676727    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.677074 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.677047    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.677135 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.677101    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.677178 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.677135    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.678895 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.678877    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.679210 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.679192    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.684628 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.684604    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqbr6\" (UniqueName: \"kubernetes.io/projected/f4005f0d-d3a4-480b-ade6-16adbc2da7b5-kube-api-access-gqbr6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-69gvn\" (UID: \"f4005f0d-d3a4-480b-ade6-16adbc2da7b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.869005 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.868919    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:35.993258 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:35.993219    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"]
Jul 26 15:47:35.995621 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:47:35.995591    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4005f0d_d3a4_480b_ade6_16adbc2da7b5.slice/crio-28705794369c917428eca390e8b3fb212b588cffd121ebb686dfc2a17fe412f3 WatchSource:0}: Error finding container 28705794369c917428eca390e8b3fb212b588cffd121ebb686dfc2a17fe412f3: Status 404 returned error can't find the container with id 28705794369c917428eca390e8b3fb212b588cffd121ebb686dfc2a17fe412f3
Jul 26 15:47:36.064164 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:36.064137    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn" event={"ID":"f4005f0d-d3a4-480b-ade6-16adbc2da7b5","Type":"ContainerStarted","Data":"28705794369c917428eca390e8b3fb212b588cffd121ebb686dfc2a17fe412f3"}
Jul 26 15:47:36.285400 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:36.284746    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:47:38.077505 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:38.077468    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn" event={"ID":"f4005f0d-d3a4-480b-ade6-16adbc2da7b5","Type":"ContainerStarted","Data":"7d197b06af629a84446a07e202a7f17065c41d1b5ddd5ec9539d8b3d293a22b9"}
Jul 26 15:47:38.094344 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:38.094306    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn" podStartSLOduration=2.076849429 podStartE2EDuration="3.094290688s" totalImagesPullingTime="1.017441259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:47:38.093158387 +0000 UTC m=+557.277224546" watchObservedRunningTime="2026-07-26 15:47:38.094290688 +0000 UTC m=+557.278356826"
Jul 26 15:47:45.869967 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:45.869935    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:45.870516 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:45.869982    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:45.882462 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:45.882439    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:46.126045 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:46.125967    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-69gvn"
Jul 26 15:47:53.395725 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:47:53.395686    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:48:00.184421 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:00.184388    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:48:06.060153 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.060123    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"]
Jul 26 15:48:06.063615 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.063594    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.065792 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.065768    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 26 15:48:06.073617 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.073583    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"]
Jul 26 15:48:06.153593 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.153565    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.153756 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.153598    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs8kx\" (UniqueName: \"kubernetes.io/projected/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-kube-api-access-gs8kx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.153756 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.153626    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.153832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.153768    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.153832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.153800    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.153832 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.153822    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.254886 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.254853    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255031 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.254894    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255031 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.254920    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255031 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.254966    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255031 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.254986    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs8kx\" (UniqueName: \"kubernetes.io/projected/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-kube-api-access-gs8kx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255210 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.255102    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255457 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.255431    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.255467    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.255569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.255502    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.257105 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.257085    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.257283 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.257265    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.262476 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.262455    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs8kx\" (UniqueName: \"kubernetes.io/projected/0f8b32a3-894e-4000-9bfa-4c2d442f0f63-kube-api-access-gs8kx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-vwm4d\" (UID: \"0f8b32a3-894e-4000-9bfa-4c2d442f0f63\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.374144 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.374061    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:06.504525 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:06.504498    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"]
Jul 26 15:48:06.506654 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:48:06.506626    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8b32a3_894e_4000_9bfa_4c2d442f0f63.slice/crio-80384c398dae58c2218a3067f2036b5a798b2d183b65c3c1c00fa33ea87fbdde WatchSource:0}: Error finding container 80384c398dae58c2218a3067f2036b5a798b2d183b65c3c1c00fa33ea87fbdde: Status 404 returned error can't find the container with id 80384c398dae58c2218a3067f2036b5a798b2d183b65c3c1c00fa33ea87fbdde
Jul 26 15:48:07.210297 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:07.210259    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d" event={"ID":"0f8b32a3-894e-4000-9bfa-4c2d442f0f63","Type":"ContainerStarted","Data":"4f1f33357b6ddd8ae8452852d825b4111bf179788b0712afed890e99859706ca"}
Jul 26 15:48:07.210639 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:07.210309    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d" event={"ID":"0f8b32a3-894e-4000-9bfa-4c2d442f0f63","Type":"ContainerStarted","Data":"80384c398dae58c2218a3067f2036b5a798b2d183b65c3c1c00fa33ea87fbdde"}
Jul 26 15:48:07.227662 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:07.227616    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d" podStartSLOduration=1.069074322 podStartE2EDuration="1.227590006s" totalImagesPullingTime="158.515684ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:48:07.226481582 +0000 UTC m=+586.410547718" watchObservedRunningTime="2026-07-26 15:48:07.227590006 +0000 UTC m=+586.411656143"
Jul 26 15:48:08.587577 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:08.587534    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:48:13.470422 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.470392    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"]
Jul 26 15:48:13.476532 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.476512    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.479013 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.478991    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 26 15:48:13.485089 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.485068    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"]
Jul 26 15:48:13.525970 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.525935    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33625ef-d0ef-4d77-8618-6fba59b5a348-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.526132 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.525996    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.526132 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.526053    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb88t\" (UniqueName: \"kubernetes.io/projected/f33625ef-d0ef-4d77-8618-6fba59b5a348-kube-api-access-rb88t\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.526281 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.526148    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.526281 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.526186    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.526281 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.526222    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.627590 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.627558    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.627756 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.627598    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.627756 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.627652    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33625ef-d0ef-4d77-8618-6fba59b5a348-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.627756 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.627714    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.627756 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.627739    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb88t\" (UniqueName: \"kubernetes.io/projected/f33625ef-d0ef-4d77-8618-6fba59b5a348-kube-api-access-rb88t\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.627970 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.627813    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.628102 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.628082    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.628184 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.628162    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.628223 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.628197    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.629919 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.629898    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f33625ef-d0ef-4d77-8618-6fba59b5a348-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.630006 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.629993    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f33625ef-d0ef-4d77-8618-6fba59b5a348-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.635323 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.635303    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb88t\" (UniqueName: \"kubernetes.io/projected/f33625ef-d0ef-4d77-8618-6fba59b5a348-kube-api-access-rb88t\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-9np9g\" (UID: \"f33625ef-d0ef-4d77-8618-6fba59b5a348\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.786527 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.786446    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:13.913023 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:13.912998    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"]
Jul 26 15:48:13.914830 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:48:13.914803    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf33625ef_d0ef_4d77_8618_6fba59b5a348.slice/crio-fdbf7bfd46e1b89f5d328a4b6f11d16bf7ea88f413ef71f73cf0349eb2ebf116 WatchSource:0}: Error finding container fdbf7bfd46e1b89f5d328a4b6f11d16bf7ea88f413ef71f73cf0349eb2ebf116: Status 404 returned error can't find the container with id fdbf7bfd46e1b89f5d328a4b6f11d16bf7ea88f413ef71f73cf0349eb2ebf116
Jul 26 15:48:14.242711 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:14.242676    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g" event={"ID":"f33625ef-d0ef-4d77-8618-6fba59b5a348","Type":"ContainerStarted","Data":"ca6d40b63cd894aa0c61a1c1e6633eb2e8588f6f3337d91f538dc70b66df8c2b"}
Jul 26 15:48:14.242875 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:14.242727    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g" event={"ID":"f33625ef-d0ef-4d77-8618-6fba59b5a348","Type":"ContainerStarted","Data":"fdbf7bfd46e1b89f5d328a4b6f11d16bf7ea88f413ef71f73cf0349eb2ebf116"}
Jul 26 15:48:14.260015 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:14.259968    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g" podStartSLOduration=1.098749826 podStartE2EDuration="1.259953169s" totalImagesPullingTime="161.203343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:48:14.257973153 +0000 UTC m=+593.442039289" watchObservedRunningTime="2026-07-26 15:48:14.259953169 +0000 UTC m=+593.444019306"
Jul 26 15:48:16.374431 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:16.374398    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:16.374838 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:16.374453    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:16.387506 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:16.387481    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:17.269508 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:17.269481    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-vwm4d"
Jul 26 15:48:21.283609 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:21.283576    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:48:23.786672 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:23.786638    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:23.787052 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:23.786683    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:23.798865 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:23.798846    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:48:24.299855 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:48:24.299823    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-9np9g"
Jul 26 15:49:29.788195 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:49:29.788155    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:49:41.187569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:49:41.187531    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:49:50.592738 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:49:50.592702    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:51:34.003243 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:51:34.003193    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:51:43.496879 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:51:43.496788    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:52:11.484539 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:52:11.484505    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:52:21.295279 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:52:21.295252    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:52:31.685812 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:52:31.685773    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:52:41.282523 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:52:41.282483    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:52:50.797331 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:52:50.797297    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:53:01.183695 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:53:01.183658    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:53:13.391435 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:53:13.391359    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:53:23.484688 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:53:23.484654    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:53:44.691445 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:53:44.691413    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:53:55.083569 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:53:55.083531    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:54:18.391986 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:54:18.391944    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:54:31.092662 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:54:31.092629    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:54:45.196072 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:54:45.195995    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:54:56.789743 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:54:56.789708    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:55:11.399249 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:55:11.399207    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:57:12.999430 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:57:12.999393    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:57:25.093619 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:57:25.093581    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:57:33.995407 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:57:33.995372    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:57:45.788306 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:57:45.788218    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:57:55.487271 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:57:55.486269    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:58:06.386688 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:58:06.386659    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:58:16.586807 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:58:16.586774    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:58:26.095257 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:58:26.093259    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:58:36.893360 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:58:36.893322    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:58:47.299086 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:58:47.299057    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:59:00.884470 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:00.884432    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:59:18.797503 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:18.797417    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:59:28.690174 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:28.690142    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:59:31.176117 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.175998    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"]
Jul 26 15:59:31.180122 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.180104    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.182494 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.182472    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-25ae8ef7-kserve-self-signed-certs\""
Jul 26 15:59:31.198962 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.198938    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"]
Jul 26 15:59:31.207672 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.207651    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-dshm\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.207765 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.207691    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tmp-dir\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.207765 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.207715    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-model-cache\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.207887 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.207866    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-home\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.207945 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.207916    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tls-certs\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.208002 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.207954    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fslrc\" (UniqueName: \"kubernetes.io/projected/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-kube-api-access-fslrc\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309217 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309189    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tmp-dir\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309382 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309323    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-model-cache\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309450 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309396    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-home\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309514 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309468    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tls-certs\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309514 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309505    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fslrc\" (UniqueName: \"kubernetes.io/projected/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-kube-api-access-fslrc\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309651 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309628    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tmp-dir\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309715 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309659    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-dshm\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309715 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309704    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-model-cache\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.309842 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.309740    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-home\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.311803 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.311775    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-dshm\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.311945 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.311926    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tls-certs\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.328171 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.328150    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fslrc\" (UniqueName: \"kubernetes.io/projected/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-kube-api-access-fslrc\") pod \"e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") " pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.489540 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.489500    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:31.622866 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:31.622840    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"]
Jul 26 15:59:31.625481 ip-10-0-138-109 kubenswrapper[2534]: W0726 15:59:31.625455    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9e2af2c_0284_4fc5_8200_0f6fc3ef7d41.slice/crio-b83f1e2b1a79f3571c811a72945cfccd62e85f52c1fa8726f1eb53c4fd5d7362 WatchSource:0}: Error finding container b83f1e2b1a79f3571c811a72945cfccd62e85f52c1fa8726f1eb53c4fd5d7362: Status 404 returned error can't find the container with id b83f1e2b1a79f3571c811a72945cfccd62e85f52c1fa8726f1eb53c4fd5d7362
Jul 26 15:59:32.403114 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:32.403080    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp" event={"ID":"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41","Type":"ContainerStarted","Data":"44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df"}
Jul 26 15:59:32.403486 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:32.403125    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp" event={"ID":"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41","Type":"ContainerStarted","Data":"b83f1e2b1a79f3571c811a72945cfccd62e85f52c1fa8726f1eb53c4fd5d7362"}
Jul 26 15:59:32.420519 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:32.420473    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp" podStartSLOduration=1.42045993 podStartE2EDuration="1.42045993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 15:59:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 15:59:32.418088543 +0000 UTC m=+1271.602154786" watchObservedRunningTime="2026-07-26 15:59:32.42045993 +0000 UTC m=+1271.604526087"
Jul 26 15:59:41.489946 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:41.489915    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:41.490381 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:41.489956    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:41.502695 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:41.502676    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:42.460817 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:42.460786    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 15:59:43.790605 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:43.790567    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 15:59:54.784593 ip-10-0-138-109 kubenswrapper[2534]: I0726 15:59:54.784557    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:00:09.377225 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:09.377185    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"]
Jul 26 16:00:13.798445 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:13.798408    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:00:24.396304 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.396243    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp" podUID="e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" containerName="main" containerID="cri-o://44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df" gracePeriod=45
Jul 26 16:00:24.637881 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.637856    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 16:00:24.644267 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.644217    2534 generic.go:356] "Generic (PLEG): container finished" podID="e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" containerID="44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df" exitCode=0
Jul 26 16:00:24.644383 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.644280    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"
Jul 26 16:00:24.644383 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.644307    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp" event={"ID":"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41","Type":"ContainerDied","Data":"44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df"}
Jul 26 16:00:24.644383 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.644370    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp" event={"ID":"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41","Type":"ContainerDied","Data":"b83f1e2b1a79f3571c811a72945cfccd62e85f52c1fa8726f1eb53c4fd5d7362"}
Jul 26 16:00:24.644491 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.644399    2534 scope.go:117] "RemoveContainer" containerID="44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df"
Jul 26 16:00:24.654758 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.654725    2534 scope.go:117] "RemoveContainer" containerID="44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df"
Jul 26 16:00:24.655047 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:00:24.655025    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df\": container with ID starting with 44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df not found: ID does not exist" containerID="44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df"
Jul 26 16:00:24.655102 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.655050    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df"} err="failed to get container status \"44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df\": rpc error: code = NotFound desc = could not find container \"44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df\": container with ID starting with 44f85fca68791931bc21b19c561333610d37fed7f5de73ebce6e6ae436f661df not found: ID does not exist"
Jul 26 16:00:24.688767 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688745    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-model-cache\") pod \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") "
Jul 26 16:00:24.688887 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688811    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fslrc\" (UniqueName: \"kubernetes.io/projected/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-kube-api-access-fslrc\") pod \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") "
Jul 26 16:00:24.688887 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688847    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-home\") pod \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") "
Jul 26 16:00:24.688995 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688901    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-dshm\") pod \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") "
Jul 26 16:00:24.688995 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688938    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tls-certs\") pod \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") "
Jul 26 16:00:24.688995 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688960    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tmp-dir\") pod \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\" (UID: \"e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41\") "
Jul 26 16:00:24.689145 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.688994    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-model-cache" (OuterVolumeSpecName: "model-cache") pod "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" (UID: "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:24.689145 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.689091    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-home" (OuterVolumeSpecName: "home") pod "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" (UID: "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:24.689281 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.689264    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:00:24.689350 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.689288    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:00:24.689350 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.689294    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" (UID: "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:24.690863 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.690838    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-kube-api-access-fslrc" (OuterVolumeSpecName: "kube-api-access-fslrc") pod "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" (UID: "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41"). InnerVolumeSpecName "kube-api-access-fslrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:00:24.690957 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.690912    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-dshm" (OuterVolumeSpecName: "dshm") pod "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" (UID: "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:00:24.691015 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.690980    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" (UID: "e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:00:24.790464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.790432    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fslrc\" (UniqueName: \"kubernetes.io/projected/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-kube-api-access-fslrc\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:00:24.790464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.790460    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:00:24.790464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.790471    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:00:24.790693 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.790479    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:00:24.965099 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.965068    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"]
Jul 26 16:00:24.975571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:24.973709    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-25ae8ef7-kserve-5d755b756c-kc2xp"]
Jul 26 16:00:25.294221 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:25.294142    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" path="/var/lib/kubelet/pods/e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41/volumes"
Jul 26 16:00:28.288322 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:28.288284    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:00:37.296629 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:37.296596    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:00:46.886852 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:00:46.886765    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:01:00.188931 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:01:00.188892    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:01:10.191540 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:01:10.191504    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:01:20.496104 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:01:20.496065    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:01:29.986911 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:01:29.986879    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:01:40.889344 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:01:40.889309    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:01:52.685401 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:01:52.685366    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:02:08.884000 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:02:08.883966    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:02:20.290850 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:02:20.290774    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:02:33.088883 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:02:33.088852    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:02:42.292474 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:02:42.292441    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:02:54.693398 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:02:54.693360    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:03:06.893553 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:03:06.893518    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:03:16.388854 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:03:16.388824    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:03:28.483365 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:03:28.483330    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:03:44.095064 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:03:44.094982    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:03:57.983569 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:03:57.983531    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:04:31.579678 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:04:31.579642    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:04:42.986785 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:04:42.986752    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:05:15.489731 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:05:15.489650    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:05:26.588490 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:05:26.588455    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:05:58.286915 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:05:58.286880    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:06:10.490530 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:06:10.490491    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:06:43.392999 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:06:43.392924    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:08:10.099430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:08:10.099394    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:08:21.488874 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:08:21.488835    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:08:31.993880 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:08:31.993846    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:08:50.687773 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:08:50.687735    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:09:26.290141 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:26.290107    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:09:35.781896 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.781856    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"]
Jul 26 16:09:35.782456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.782437    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" containerName="main"
Jul 26 16:09:35.782538 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.782459    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" containerName="main"
Jul 26 16:09:35.782592 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.782585    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9e2af2c-0284-4fc5-8200-0f6fc3ef7d41" containerName="main"
Jul 26 16:09:35.785546 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.785525    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.788239 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.788208    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1a44779b\"/\"kube-root-ca.crt\""
Jul 26 16:09:35.788323 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.788309    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1a44779b\"/\"default-dockercfg-4kppl\""
Jul 26 16:09:35.789282 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.789265    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1a44779b\"/\"openshift-service-ca.crt\""
Jul 26 16:09:35.789349 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.789279    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1a44779b\"/\"e2e-lifecycle-model-1a44779b-kserve-self-signed-certs\""
Jul 26 16:09:35.803600 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.803578    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"]
Jul 26 16:09:35.880563 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.880537    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tmp-dir\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.880692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.880582    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-dshm\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.880692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.880615    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tls-certs\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.880692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.880640    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-model-cache\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.880692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.880657    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-home\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.880890 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.880756    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t9mq\" (UniqueName: \"kubernetes.io/projected/37a1bccb-07f8-4d78-9106-ac841cc00dd1-kube-api-access-9t9mq\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.981564 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.981526    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-home\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.981746 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.981583    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t9mq\" (UniqueName: \"kubernetes.io/projected/37a1bccb-07f8-4d78-9106-ac841cc00dd1-kube-api-access-9t9mq\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.981746 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.981631    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tmp-dir\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.981746 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.981664    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-dshm\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.981746 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.981688    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tls-certs\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.981746 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.981726    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-model-cache\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.982037 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.982005    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-home\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.982078 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.982058    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tmp-dir\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.982172 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.982155    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-model-cache\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.983946 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.983925    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-dshm\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.984187 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.984169    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tls-certs\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:35.990156 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:35.990137    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t9mq\" (UniqueName: \"kubernetes.io/projected/37a1bccb-07f8-4d78-9106-ac841cc00dd1-kube-api-access-9t9mq\") pod \"e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") " pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:36.094861 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:36.094787    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:36.227568 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:36.227541    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"]
Jul 26 16:09:36.229793 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:09:36.229724    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37a1bccb_07f8_4d78_9106_ac841cc00dd1.slice/crio-08279d7c3fe2d1746e09af6ba24d6cd17f563bde4a0f665553ab2f8e3c4602c3 WatchSource:0}: Error finding container 08279d7c3fe2d1746e09af6ba24d6cd17f563bde4a0f665553ab2f8e3c4602c3: Status 404 returned error can't find the container with id 08279d7c3fe2d1746e09af6ba24d6cd17f563bde4a0f665553ab2f8e3c4602c3
Jul 26 16:09:37.211389 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:37.211351    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb" event={"ID":"37a1bccb-07f8-4d78-9106-ac841cc00dd1","Type":"ContainerStarted","Data":"aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765"}
Jul 26 16:09:37.211389 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:37.211393    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb" event={"ID":"37a1bccb-07f8-4d78-9106-ac841cc00dd1","Type":"ContainerStarted","Data":"08279d7c3fe2d1746e09af6ba24d6cd17f563bde4a0f665553ab2f8e3c4602c3"}
Jul 26 16:09:37.229506 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:37.229459    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb" podStartSLOduration=2.229444673 podStartE2EDuration="2.229444673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:09:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:09:37.227667981 +0000 UTC m=+1876.411734120" watchObservedRunningTime="2026-07-26 16:09:37.229444673 +0000 UTC m=+1876.413510810"
Jul 26 16:09:40.885984 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:40.885948    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:09:46.095003 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:46.094914    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:46.095003 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:46.094954    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:46.108005 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:46.107982    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:46.264140 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:46.264111    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:09:55.885152 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:09:55.885122    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:10:14.293743 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:14.293698    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:10:22.885205 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:22.885168    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:10:28.218063 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:28.218025    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"]
Jul 26 16:10:43.237302 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.237224    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb" podUID="37a1bccb-07f8-4d78-9106-ac841cc00dd1" containerName="main" containerID="cri-o://aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765" gracePeriod=45
Jul 26 16:10:43.491181 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.491124    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:10:43.523492 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.523448    2534 generic.go:356] "Generic (PLEG): container finished" podID="37a1bccb-07f8-4d78-9106-ac841cc00dd1" containerID="aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765" exitCode=0
Jul 26 16:10:43.523664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.523509    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"
Jul 26 16:10:43.523664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.523541    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb" event={"ID":"37a1bccb-07f8-4d78-9106-ac841cc00dd1","Type":"ContainerDied","Data":"aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765"}
Jul 26 16:10:43.523664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.523585    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb" event={"ID":"37a1bccb-07f8-4d78-9106-ac841cc00dd1","Type":"ContainerDied","Data":"08279d7c3fe2d1746e09af6ba24d6cd17f563bde4a0f665553ab2f8e3c4602c3"}
Jul 26 16:10:43.523664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.523606    2534 scope.go:117] "RemoveContainer" containerID="aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765"
Jul 26 16:10:43.532906 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.532872    2534 scope.go:117] "RemoveContainer" containerID="aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765"
Jul 26 16:10:43.533175 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:10:43.533154    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765\": container with ID starting with aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765 not found: ID does not exist" containerID="aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765"
Jul 26 16:10:43.533291 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.533182    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765"} err="failed to get container status \"aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765\": rpc error: code = NotFound desc = could not find container \"aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765\": container with ID starting with aa209294c0f92ad6e08bf1bff5ca563c3c091632b8324dcc9984a6654224b765 not found: ID does not exist"
Jul 26 16:10:43.576678 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576651    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-model-cache\") pod \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") "
Jul 26 16:10:43.576774 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576691    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-dshm\") pod \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") "
Jul 26 16:10:43.576774 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576717    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-home\") pod \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") "
Jul 26 16:10:43.576774 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576744    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9t9mq\" (UniqueName: \"kubernetes.io/projected/37a1bccb-07f8-4d78-9106-ac841cc00dd1-kube-api-access-9t9mq\") pod \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") "
Jul 26 16:10:43.576929 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576808    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tls-certs\") pod \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") "
Jul 26 16:10:43.576929 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576846    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tmp-dir\") pod \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\" (UID: \"37a1bccb-07f8-4d78-9106-ac841cc00dd1\") "
Jul 26 16:10:43.576929 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576876    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-model-cache" (OuterVolumeSpecName: "model-cache") pod "37a1bccb-07f8-4d78-9106-ac841cc00dd1" (UID: "37a1bccb-07f8-4d78-9106-ac841cc00dd1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:10:43.577076 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.576994    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-home" (OuterVolumeSpecName: "home") pod "37a1bccb-07f8-4d78-9106-ac841cc00dd1" (UID: "37a1bccb-07f8-4d78-9106-ac841cc00dd1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:10:43.577168 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.577147    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37a1bccb-07f8-4d78-9106-ac841cc00dd1" (UID: "37a1bccb-07f8-4d78-9106-ac841cc00dd1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:10:43.577224 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.577181    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:10:43.577224 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.577201    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:10:43.578907 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.578879    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-dshm" (OuterVolumeSpecName: "dshm") pod "37a1bccb-07f8-4d78-9106-ac841cc00dd1" (UID: "37a1bccb-07f8-4d78-9106-ac841cc00dd1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:10:43.578907 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.578882    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37a1bccb-07f8-4d78-9106-ac841cc00dd1" (UID: "37a1bccb-07f8-4d78-9106-ac841cc00dd1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:10:43.579022 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.578930    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a1bccb-07f8-4d78-9106-ac841cc00dd1-kube-api-access-9t9mq" (OuterVolumeSpecName: "kube-api-access-9t9mq") pod "37a1bccb-07f8-4d78-9106-ac841cc00dd1" (UID: "37a1bccb-07f8-4d78-9106-ac841cc00dd1"). InnerVolumeSpecName "kube-api-access-9t9mq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:10:43.677731 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.677702    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:10:43.677731 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.677726    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9t9mq\" (UniqueName: \"kubernetes.io/projected/37a1bccb-07f8-4d78-9106-ac841cc00dd1-kube-api-access-9t9mq\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:10:43.677731 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.677736    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:10:43.677934 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.677745    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37a1bccb-07f8-4d78-9106-ac841cc00dd1-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:10:43.849703 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.847332    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"]
Jul 26 16:10:43.855761 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:43.853403    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1a44779b/e2e-lifecycle-model-1a44779b-kserve-9ccfdc8c9-gtrcb"]
Jul 26 16:10:45.294837 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:10:45.294804    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37a1bccb-07f8-4d78-9106-ac841cc00dd1" path="/var/lib/kubelet/pods/37a1bccb-07f8-4d78-9106-ac841cc00dd1/volumes"
Jul 26 16:11:15.394447 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:11:15.394378    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:11:27.808970 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:11:27.808933    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:12:16.483221 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:16.483187    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:12:31.281751 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.281710    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:12:31.604072 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.604001    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"]
Jul 26 16:12:31.604395 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.604382    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a1bccb-07f8-4d78-9106-ac841cc00dd1" containerName="main"
Jul 26 16:12:31.604446 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.604397    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a1bccb-07f8-4d78-9106-ac841cc00dd1" containerName="main"
Jul 26 16:12:31.604496 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.604487    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="37a1bccb-07f8-4d78-9106-ac841cc00dd1" containerName="main"
Jul 26 16:12:31.607444 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.607427    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.611433 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.611222    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7c7176\"/\"openshift-service-ca.crt\""
Jul 26 16:12:31.611762 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.611739    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7c7176\"/\"kube-root-ca.crt\""
Jul 26 16:12:31.611853 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.611792    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7c7176\"/\"default-dockercfg-dx5st\""
Jul 26 16:12:31.611853 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.611815    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7c7176\"/\"test-model-7c7176-kserve-self-signed-certs\""
Jul 26 16:12:31.621532 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.621512    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"]
Jul 26 16:12:31.643070 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.643041    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tmp-dir\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.643173 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.643074    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-home\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.643173 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.643095    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-dshm\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.643273 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.643213    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58jgf\" (UniqueName: \"kubernetes.io/projected/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-kube-api-access-58jgf\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.643273 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.643250    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-model-cache\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.643357 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.643277    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tls-certs\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.743807 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.743774    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tmp-dir\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.743807 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.743810    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-home\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744048 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.743828    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-dshm\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744048 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.743906    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58jgf\" (UniqueName: \"kubernetes.io/projected/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-kube-api-access-58jgf\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744048 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.743941    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-model-cache\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744200 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.744093    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tls-certs\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744200 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.744161    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-home\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744322 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.744219    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tmp-dir\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.744507 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.744482    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-model-cache\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.746922 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.746900    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-dshm\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.747084 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.747067    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tls-certs\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.751701 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.751681    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58jgf\" (UniqueName: \"kubernetes.io/projected/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-kube-api-access-58jgf\") pod \"test-model-7c7176-kserve-776766479f-x868s\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") " pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:31.917994 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:31.917915    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:32.045664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:32.045639    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"]
Jul 26 16:12:32.048361 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:12:32.048333    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13aa9410_78fd_40fb_859d_ea32dd8c6e1c.slice/crio-0423826066c241f010556d9c21b26ca28ff294b0beaecc2f3d1531962529ea37 WatchSource:0}: Error finding container 0423826066c241f010556d9c21b26ca28ff294b0beaecc2f3d1531962529ea37: Status 404 returned error can't find the container with id 0423826066c241f010556d9c21b26ca28ff294b0beaecc2f3d1531962529ea37
Jul 26 16:12:33.028005 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:33.027965    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s" event={"ID":"13aa9410-78fd-40fb-859d-ea32dd8c6e1c","Type":"ContainerStarted","Data":"1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1"}
Jul 26 16:12:33.028399 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:33.028015    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s" event={"ID":"13aa9410-78fd-40fb-859d-ea32dd8c6e1c","Type":"ContainerStarted","Data":"0423826066c241f010556d9c21b26ca28ff294b0beaecc2f3d1531962529ea37"}
Jul 26 16:12:33.044420 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:33.044373    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s" podStartSLOduration=2.044359362 podStartE2EDuration="2.044359362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:12:33.043952417 +0000 UTC m=+2052.228018554" watchObservedRunningTime="2026-07-26 16:12:33.044359362 +0000 UTC m=+2052.228425500"
Jul 26 16:12:41.918244 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:41.918194    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:41.918652 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:41.918257    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:41.930470 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:41.930446    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:42.082598 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:42.082569    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:12:43.682952 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:12:43.682874    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:13:10.477185 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.477149    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"]
Jul 26 16:13:10.480674 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.480657    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.484072 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.484048    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-959cca\"/\"kube-root-ca.crt\""
Jul 26 16:13:10.484072 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.484059    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-959cca\"/\"test-model-959cca-kserve-self-signed-certs\""
Jul 26 16:13:10.484260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.484058    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-959cca\"/\"default-dockercfg-g57n4\""
Jul 26 16:13:10.484260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.484109    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-959cca\"/\"openshift-service-ca.crt\""
Jul 26 16:13:10.490028 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.490005    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"]
Jul 26 16:13:10.595775 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.595743    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww25n\" (UniqueName: \"kubernetes.io/projected/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-kube-api-access-ww25n\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.595926 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.595798    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-dshm\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.595926 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.595889    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-model-cache\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.596001 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.595933    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tls-certs\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.596001 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.595984    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tmp-dir\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.596065 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.596025    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-home\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.696621 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.696593    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww25n\" (UniqueName: \"kubernetes.io/projected/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-kube-api-access-ww25n\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.696738 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.696643    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-dshm\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.696738 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.696683    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-model-cache\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.696738 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.696717    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tls-certs\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.696892 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.696773    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tmp-dir\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.696892 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.696816    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-home\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.697184 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.697152    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-model-cache\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.697319 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.697155    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tmp-dir\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.697319 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.697197    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-home\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.698979 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.698963    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-dshm\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.699253 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.699219    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tls-certs\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.704509 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.704491    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww25n\" (UniqueName: \"kubernetes.io/projected/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-kube-api-access-ww25n\") pod \"test-model-959cca-kserve-7646df6d97-zgp44\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") " pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.791359 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.791283    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:10.941305 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:10.941283    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"]
Jul 26 16:13:10.943122 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:13:10.943093    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5d6bb3d_8f3c_4ea8_a2e5_874a4576cb20.slice/crio-df15751f1ee5edac0a95abb4bb749e38f051dcd497fb2cf99d71700bd0e66a0e WatchSource:0}: Error finding container df15751f1ee5edac0a95abb4bb749e38f051dcd497fb2cf99d71700bd0e66a0e: Status 404 returned error can't find the container with id df15751f1ee5edac0a95abb4bb749e38f051dcd497fb2cf99d71700bd0e66a0e
Jul 26 16:13:11.207920 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:11.207885    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44" event={"ID":"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20","Type":"ContainerStarted","Data":"1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e"}
Jul 26 16:13:11.208086 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:11.207936    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44" event={"ID":"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20","Type":"ContainerStarted","Data":"df15751f1ee5edac0a95abb4bb749e38f051dcd497fb2cf99d71700bd0e66a0e"}
Jul 26 16:13:11.225852 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:11.225797    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44" podStartSLOduration=1.225771498 podStartE2EDuration="1.225771498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:13:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:13:11.224262887 +0000 UTC m=+2090.408329027" watchObservedRunningTime="2026-07-26 16:13:11.225771498 +0000 UTC m=+2090.409837637"
Jul 26 16:13:13.589568 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:13.589535    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:13:20.792406 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:20.792371    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:20.792796 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:20.792642    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:20.805171 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:20.805146    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:21.267984 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:21.267950    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:13:26.987580 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:26.987544    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:13:51.560121 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:51.560087    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"]
Jul 26 16:13:52.505914 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:52.505877    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"]
Jul 26 16:13:56.484622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:13:56.484591    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:14:06.579206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.579146    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s" podUID="13aa9410-78fd-40fb-859d-ea32dd8c6e1c" containerName="main" containerID="cri-o://1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1" gracePeriod=45
Jul 26 16:14:06.833167 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.833113    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:14:06.901099 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901071    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tmp-dir\") pod \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") "
Jul 26 16:14:06.901226 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901116    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-home\") pod \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") "
Jul 26 16:14:06.901226 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901134    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-model-cache\") pod \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") "
Jul 26 16:14:06.901226 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901169    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tls-certs\") pod \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") "
Jul 26 16:14:06.901373 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901246    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-dshm\") pod \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") "
Jul 26 16:14:06.901373 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901276    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58jgf\" (UniqueName: \"kubernetes.io/projected/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-kube-api-access-58jgf\") pod \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\" (UID: \"13aa9410-78fd-40fb-859d-ea32dd8c6e1c\") "
Jul 26 16:14:06.901449 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901364    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "13aa9410-78fd-40fb-859d-ea32dd8c6e1c" (UID: "13aa9410-78fd-40fb-859d-ea32dd8c6e1c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:06.901449 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901383    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-home" (OuterVolumeSpecName: "home") pod "13aa9410-78fd-40fb-859d-ea32dd8c6e1c" (UID: "13aa9410-78fd-40fb-859d-ea32dd8c6e1c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:06.901519 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901456    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-model-cache" (OuterVolumeSpecName: "model-cache") pod "13aa9410-78fd-40fb-859d-ea32dd8c6e1c" (UID: "13aa9410-78fd-40fb-859d-ea32dd8c6e1c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:06.901704 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901686    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:06.901761 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901705    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:06.901761 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.901714    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:06.903430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.903382    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-dshm" (OuterVolumeSpecName: "dshm") pod "13aa9410-78fd-40fb-859d-ea32dd8c6e1c" (UID: "13aa9410-78fd-40fb-859d-ea32dd8c6e1c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:06.903527 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.903443    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-kube-api-access-58jgf" (OuterVolumeSpecName: "kube-api-access-58jgf") pod "13aa9410-78fd-40fb-859d-ea32dd8c6e1c" (UID: "13aa9410-78fd-40fb-859d-ea32dd8c6e1c"). InnerVolumeSpecName "kube-api-access-58jgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:06.903527 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:06.903458    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "13aa9410-78fd-40fb-859d-ea32dd8c6e1c" (UID: "13aa9410-78fd-40fb-859d-ea32dd8c6e1c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:14:07.003096 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.003066    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:07.003096 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.003092    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58jgf\" (UniqueName: \"kubernetes.io/projected/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-kube-api-access-58jgf\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:07.003273 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.003103    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13aa9410-78fd-40fb-859d-ea32dd8c6e1c-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:07.476959 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.476928    2534 generic.go:356] "Generic (PLEG): container finished" podID="13aa9410-78fd-40fb-859d-ea32dd8c6e1c" containerID="1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1" exitCode=0
Jul 26 16:14:07.477119 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.476974    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"
Jul 26 16:14:07.477119 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.476980    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s" event={"ID":"13aa9410-78fd-40fb-859d-ea32dd8c6e1c","Type":"ContainerDied","Data":"1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1"}
Jul 26 16:14:07.477119 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.477023    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s" event={"ID":"13aa9410-78fd-40fb-859d-ea32dd8c6e1c","Type":"ContainerDied","Data":"0423826066c241f010556d9c21b26ca28ff294b0beaecc2f3d1531962529ea37"}
Jul 26 16:14:07.477119 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.477043    2534 scope.go:117] "RemoveContainer" containerID="1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1"
Jul 26 16:14:07.485501 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.485485    2534 scope.go:117] "RemoveContainer" containerID="1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1"
Jul 26 16:14:07.485750 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:14:07.485728    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1\": container with ID starting with 1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1 not found: ID does not exist" containerID="1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1"
Jul 26 16:14:07.485827 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.485761    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1"} err="failed to get container status \"1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1\": rpc error: code = NotFound desc = could not find container \"1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1\": container with ID starting with 1c74a82476ad28fb28641c2387050d4df066ad4ec69eb2e9e18f6a481033a4f1 not found: ID does not exist"
Jul 26 16:14:07.492879 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.492855    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"]
Jul 26 16:14:07.497186 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.497164    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7c7176/test-model-7c7176-kserve-776766479f-x868s"]
Jul 26 16:14:07.525369 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.525336    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44" podUID="c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" containerName="main" containerID="cri-o://1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e" gracePeriod=45
Jul 26 16:14:07.768033 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.768009    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:14:07.909455 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909377    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-dshm\") pod \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") "
Jul 26 16:14:07.909455 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909429    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tls-certs\") pod \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") "
Jul 26 16:14:07.909455 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909450    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ww25n\" (UniqueName: \"kubernetes.io/projected/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-kube-api-access-ww25n\") pod \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") "
Jul 26 16:14:07.909744 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909488    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-model-cache\") pod \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") "
Jul 26 16:14:07.909744 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909521    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tmp-dir\") pod \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") "
Jul 26 16:14:07.909744 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909573    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-home\") pod \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\" (UID: \"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20\") "
Jul 26 16:14:07.909949 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909804    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-model-cache" (OuterVolumeSpecName: "model-cache") pod "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" (UID: "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:07.909949 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909843    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" (UID: "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:07.909949 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.909886    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-home" (OuterVolumeSpecName: "home") pod "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" (UID: "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:07.911720 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.911692    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-dshm" (OuterVolumeSpecName: "dshm") pod "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" (UID: "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:14:07.911832 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.911758    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-kube-api-access-ww25n" (OuterVolumeSpecName: "kube-api-access-ww25n") pod "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" (UID: "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20"). InnerVolumeSpecName "kube-api-access-ww25n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:14:07.911832 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:07.911794    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" (UID: "c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:14:08.010544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.010512    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:08.010544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.010539    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:08.010544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.010549    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:08.010790 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.010558    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:08.010790 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.010567    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww25n\" (UniqueName: \"kubernetes.io/projected/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-kube-api-access-ww25n\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:08.010790 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.010576    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:14:08.482792 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.482760    2534 generic.go:356] "Generic (PLEG): container finished" podID="c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" containerID="1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e" exitCode=0
Jul 26 16:14:08.482985 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.482813    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"
Jul 26 16:14:08.482985 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.482844    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44" event={"ID":"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20","Type":"ContainerDied","Data":"1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e"}
Jul 26 16:14:08.482985 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.482878    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44" event={"ID":"c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20","Type":"ContainerDied","Data":"df15751f1ee5edac0a95abb4bb749e38f051dcd497fb2cf99d71700bd0e66a0e"}
Jul 26 16:14:08.482985 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.482900    2534 scope.go:117] "RemoveContainer" containerID="1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e"
Jul 26 16:14:08.491463 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.491447    2534 scope.go:117] "RemoveContainer" containerID="1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e"
Jul 26 16:14:08.491683 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:14:08.491668    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e\": container with ID starting with 1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e not found: ID does not exist" containerID="1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e"
Jul 26 16:14:08.491723 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.491690    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e"} err="failed to get container status \"1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e\": rpc error: code = NotFound desc = could not find container \"1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e\": container with ID starting with 1c029ada5d158ca9f0321e61fef074e7d6affdc879d0b77260fba9aafd9c6d4e not found: ID does not exist"
Jul 26 16:14:08.505473 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.505447    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"]
Jul 26 16:14:08.508981 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:08.508955    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-959cca/test-model-959cca-kserve-7646df6d97-zgp44"]
Jul 26 16:14:09.094659 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:09.094629    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:14:09.294047 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:09.294014    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13aa9410-78fd-40fb-859d-ea32dd8c6e1c" path="/var/lib/kubelet/pods/13aa9410-78fd-40fb-859d-ea32dd8c6e1c/volumes"
Jul 26 16:14:09.294514 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:09.294492    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" path="/var/lib/kubelet/pods/c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20/volumes"
Jul 26 16:14:26.587672 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:26.587639    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:14:45.698152 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:14:45.698114    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:15:00.128016 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.127980    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"]
Jul 26 16:15:00.128524 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.128373    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" containerName="main"
Jul 26 16:15:00.128524 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.128385    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" containerName="main"
Jul 26 16:15:00.128524 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.128397    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13aa9410-78fd-40fb-859d-ea32dd8c6e1c" containerName="main"
Jul 26 16:15:00.128524 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.128403    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="13aa9410-78fd-40fb-859d-ea32dd8c6e1c" containerName="main"
Jul 26 16:15:00.128524 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.128480    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5d6bb3d-8f3c-4ea8-a2e5-874a4576cb20" containerName="main"
Jul 26 16:15:00.128524 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.128490    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="13aa9410-78fd-40fb-859d-ea32dd8c6e1c" containerName="main"
Jul 26 16:15:00.131490 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.131469    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"
Jul 26 16:15:00.131611 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.131486    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"]
Jul 26 16:15:00.133797 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.133770    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:15:00.134759 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.134742    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"
Jul 26 16:15:00.138339 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.138312    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"]
Jul 26 16:15:00.144002 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.143984    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"
Jul 26 16:15:00.150988 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.150969    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"
Jul 26 16:15:00.153907 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.153887    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"]
Jul 26 16:15:00.314105 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.314083    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"]
Jul 26 16:15:00.317009 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:15:00.316983    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81205460_7602_4ec5_be8a_029461f89b5d.slice/crio-ca8146978fb47f41c739ca44f6c17e7eeaa964326bcf3cdb981b998be19d4727 WatchSource:0}: Error finding container ca8146978fb47f41c739ca44f6c17e7eeaa964326bcf3cdb981b998be19d4727: Status 404 returned error can't find the container with id ca8146978fb47f41c739ca44f6c17e7eeaa964326bcf3cdb981b998be19d4727
Jul 26 16:15:00.319243 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.319219    2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:15:00.332924 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.332862    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"]
Jul 26 16:15:00.334853 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:15:00.334827    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode464877f_f22b_40c1_93f1_ca79d25ec5e3.slice/crio-522f12f783ef938fa5bc105ed57d906d6bb7f56104518221845d6715cbb1c542 WatchSource:0}: Error finding container 522f12f783ef938fa5bc105ed57d906d6bb7f56104518221845d6715cbb1c542: Status 404 returned error can't find the container with id 522f12f783ef938fa5bc105ed57d906d6bb7f56104518221845d6715cbb1c542
Jul 26 16:15:00.729307 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.729273    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48" event={"ID":"81205460-7602-4ec5-be8a-029461f89b5d","Type":"ContainerStarted","Data":"ca8146978fb47f41c739ca44f6c17e7eeaa964326bcf3cdb981b998be19d4727"}
Jul 26 16:15:00.730459 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:00.730438    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925" event={"ID":"e464877f-f22b-40c1-93f1-ca79d25ec5e3","Type":"ContainerStarted","Data":"522f12f783ef938fa5bc105ed57d906d6bb7f56104518221845d6715cbb1c542"}
Jul 26 16:15:02.789176 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:02.789140    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:15:03.758066 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:03.758025    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48" event={"ID":"81205460-7602-4ec5-be8a-029461f89b5d","Type":"ContainerStarted","Data":"f7334bd9e9dae86dc814be24e1cb09175a53117d79a474240baab211537c0d24"}
Jul 26 16:15:03.759585 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:03.759556    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925" event={"ID":"e464877f-f22b-40c1-93f1-ca79d25ec5e3","Type":"ContainerStarted","Data":"565dbb7f05de51b21b1aa12d241709ed247372267af6e82aadcb79e22c741203"}
Jul 26 16:15:03.774016 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:03.773976    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48" podStartSLOduration=0.988338916 podStartE2EDuration="3.773962697s" totalImagesPullingTime="2.785623781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:15:03.77292745 +0000 UTC m=+2202.956993586" watchObservedRunningTime="2026-07-26 16:15:03.773962697 +0000 UTC m=+2202.958028871"
Jul 26 16:15:03.788527 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:03.788485    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925" podStartSLOduration=1.02446983 podStartE2EDuration="3.788472972s" totalImagesPullingTime="2.764003142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:15:03.787506227 +0000 UTC m=+2202.971572353" watchObservedRunningTime="2026-07-26 16:15:03.788472972 +0000 UTC m=+2202.972539108"
Jul 26 16:15:04.765928 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:04.765892    2534 generic.go:356] "Generic (PLEG): container finished" podID="81205460-7602-4ec5-be8a-029461f89b5d" containerID="f7334bd9e9dae86dc814be24e1cb09175a53117d79a474240baab211537c0d24" exitCode=0
Jul 26 16:15:04.766379 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:04.765975    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48" event={"ID":"81205460-7602-4ec5-be8a-029461f89b5d","Type":"ContainerDied","Data":"f7334bd9e9dae86dc814be24e1cb09175a53117d79a474240baab211537c0d24"}
Jul 26 16:15:04.767456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:04.767433    2534 generic.go:356] "Generic (PLEG): container finished" podID="e464877f-f22b-40c1-93f1-ca79d25ec5e3" containerID="565dbb7f05de51b21b1aa12d241709ed247372267af6e82aadcb79e22c741203" exitCode=0
Jul 26 16:15:04.767573 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:04.767487    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925" event={"ID":"e464877f-f22b-40c1-93f1-ca79d25ec5e3","Type":"ContainerDied","Data":"565dbb7f05de51b21b1aa12d241709ed247372267af6e82aadcb79e22c741203"}
Jul 26 16:15:05.911612 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:05.911586    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"
Jul 26 16:15:05.914595 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:05.914577    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"
Jul 26 16:15:06.777289 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:06.777260    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"
Jul 26 16:15:06.777476 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:06.777292    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925" event={"ID":"e464877f-f22b-40c1-93f1-ca79d25ec5e3","Type":"ContainerDied","Data":"522f12f783ef938fa5bc105ed57d906d6bb7f56104518221845d6715cbb1c542"}
Jul 26 16:15:06.777476 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:06.777342    2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="522f12f783ef938fa5bc105ed57d906d6bb7f56104518221845d6715cbb1c542"
Jul 26 16:15:06.778636 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:06.778611    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48" event={"ID":"81205460-7602-4ec5-be8a-029461f89b5d","Type":"ContainerDied","Data":"ca8146978fb47f41c739ca44f6c17e7eeaa964326bcf3cdb981b998be19d4727"}
Jul 26 16:15:06.778636 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:06.778635    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751375-bpq48"
Jul 26 16:15:06.778771 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:06.778658    2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca8146978fb47f41c739ca44f6c17e7eeaa964326bcf3cdb981b998be19d4727"
Jul 26 16:15:15.833947 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:15.833917    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"]
Jul 26 16:15:15.839633 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:15.839609    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-4b0d96-29751375-7z925"]
Jul 26 16:15:17.293919 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:17.293886    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e464877f-f22b-40c1-93f1-ca79d25ec5e3" path="/var/lib/kubelet/pods/e464877f-f22b-40c1-93f1-ca79d25ec5e3/volumes"
Jul 26 16:15:19.895328 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:19.895289    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:15:27.142833 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.142797    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"]
Jul 26 16:15:27.143203 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.143179    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81205460-7602-4ec5-be8a-029461f89b5d" containerName="cleanup"
Jul 26 16:15:27.143203 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.143191    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="81205460-7602-4ec5-be8a-029461f89b5d" containerName="cleanup"
Jul 26 16:15:27.143203 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.143202    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e464877f-f22b-40c1-93f1-ca79d25ec5e3" containerName="cleanup"
Jul 26 16:15:27.143323 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.143208    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e464877f-f22b-40c1-93f1-ca79d25ec5e3" containerName="cleanup"
Jul 26 16:15:27.143323 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.143311    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e464877f-f22b-40c1-93f1-ca79d25ec5e3" containerName="cleanup"
Jul 26 16:15:27.143323 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.143322    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="81205460-7602-4ec5-be8a-029461f89b5d" containerName="cleanup"
Jul 26 16:15:27.146396 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.146380    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:27.148676 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.148659    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:15:27.148732 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.148657    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:15:27.153171 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.153148    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"]
Jul 26 16:15:27.227017 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.226982    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c08d5a-5aa9-42e4-98c2-73e661f37135-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h\" (UID: \"e9c08d5a-5aa9-42e4-98c2-73e661f37135\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:27.328301 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.328270    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c08d5a-5aa9-42e4-98c2-73e661f37135-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h\" (UID: \"e9c08d5a-5aa9-42e4-98c2-73e661f37135\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:27.328918 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.328893    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c08d5a-5aa9-42e4-98c2-73e661f37135-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h\" (UID: \"e9c08d5a-5aa9-42e4-98c2-73e661f37135\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:27.458619 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.458588    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:27.796550 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.796526    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"]
Jul 26 16:15:27.798885 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:15:27.798854    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9c08d5a_5aa9_42e4_98c2_73e661f37135.slice/crio-e49d368617fe149ac29098280edaf8d3d2e8bf680d40226e9487de57aa2835f4 WatchSource:0}: Error finding container e49d368617fe149ac29098280edaf8d3d2e8bf680d40226e9487de57aa2835f4: Status 404 returned error can't find the container with id e49d368617fe149ac29098280edaf8d3d2e8bf680d40226e9487de57aa2835f4
Jul 26 16:15:27.882646 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:27.882613    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h" event={"ID":"e9c08d5a-5aa9-42e4-98c2-73e661f37135","Type":"ContainerStarted","Data":"e49d368617fe149ac29098280edaf8d3d2e8bf680d40226e9487de57aa2835f4"}
Jul 26 16:15:28.889110 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:28.889022    2534 generic.go:356] "Generic (PLEG): container finished" podID="e9c08d5a-5aa9-42e4-98c2-73e661f37135" containerID="9ec5e8b45632eaa096c9a39bb805d5256718d99f671c72d43f6c83ddbbb085bf" exitCode=0
Jul 26 16:15:28.889110 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:28.889067    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h" event={"ID":"e9c08d5a-5aa9-42e4-98c2-73e661f37135","Type":"ContainerDied","Data":"9ec5e8b45632eaa096c9a39bb805d5256718d99f671c72d43f6c83ddbbb085bf"}
Jul 26 16:15:30.030059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.030024    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:30.154175 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.154096    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c08d5a-5aa9-42e4-98c2-73e661f37135-maas-api-service-ca\") pod \"e9c08d5a-5aa9-42e4-98c2-73e661f37135\" (UID: \"e9c08d5a-5aa9-42e4-98c2-73e661f37135\") "
Jul 26 16:15:30.154445 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.154420    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9c08d5a-5aa9-42e4-98c2-73e661f37135-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9c08d5a-5aa9-42e4-98c2-73e661f37135" (UID: "e9c08d5a-5aa9-42e4-98c2-73e661f37135"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:15:30.255813 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.255768    2534 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9c08d5a-5aa9-42e4-98c2-73e661f37135-maas-api-service-ca\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:15:30.900076 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.900046    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"
Jul 26 16:15:30.900223 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.900087    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h" event={"ID":"e9c08d5a-5aa9-42e4-98c2-73e661f37135","Type":"ContainerDied","Data":"e49d368617fe149ac29098280edaf8d3d2e8bf680d40226e9487de57aa2835f4"}
Jul 26 16:15:30.900223 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:30.900130    2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e49d368617fe149ac29098280edaf8d3d2e8bf680d40226e9487de57aa2835f4"
Jul 26 16:15:47.224187 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:47.224108    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"]
Jul 26 16:15:47.226738 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:47.226712    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7cc89d-hn88h"]
Jul 26 16:15:47.293981 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:47.293952    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9c08d5a-5aa9-42e4-98c2-73e661f37135" path="/var/lib/kubelet/pods/e9c08d5a-5aa9-42e4-98c2-73e661f37135/volumes"
Jul 26 16:15:47.890954 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:15:47.890918    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:16:02.586407 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:02.586373    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:16:18.685119 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:18.685078    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:16:23.375586 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.375553    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"]
Jul 26 16:16:23.375967 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.375946    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9c08d5a-5aa9-42e4-98c2-73e661f37135" containerName="revoke-keys"
Jul 26 16:16:23.375967 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.375956    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9c08d5a-5aa9-42e4-98c2-73e661f37135" containerName="revoke-keys"
Jul 26 16:16:23.376038 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.376006    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9c08d5a-5aa9-42e4-98c2-73e661f37135" containerName="revoke-keys"
Jul 26 16:16:23.379533 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.379516    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.383399 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.383372    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aadfb2\"/\"kube-root-ca.crt\""
Jul 26 16:16:23.384176 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.384152    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aadfb2\"/\"default-dockercfg-xb4mq\""
Jul 26 16:16:23.384288 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.384180    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aadfb2\"/\"openshift-service-ca.crt\""
Jul 26 16:16:23.384288 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.384182    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aadfb2\"/\"auth-test-model-aadfb2-kserve-self-signed-certs\""
Jul 26 16:16:23.399253 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.399208    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"]
Jul 26 16:16:23.459889 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.459859    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tmp-dir\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.459889 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.459892    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-model-cache\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.460132 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.459997    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-dshm\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.460132 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.460019    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tls-certs\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.460224 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.460196    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gpsz\" (UniqueName: \"kubernetes.io/projected/7450dd57-c0f7-47d6-ab28-108ea343d5fd-kube-api-access-9gpsz\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.460340 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.460320    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-home\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561190 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561157    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tmp-dir\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561190 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561191    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-model-cache\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561411 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561213    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-dshm\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561449 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561433    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tls-certs\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561557 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561538    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gpsz\" (UniqueName: \"kubernetes.io/projected/7450dd57-c0f7-47d6-ab28-108ea343d5fd-kube-api-access-9gpsz\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561670 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561648    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tmp-dir\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561733 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561667    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-model-cache\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.561733 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.561706    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-home\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.562069 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.562051    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-home\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.563498 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.563479    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-dshm\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.563777 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.563760    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tls-certs\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.570029 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.570007    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gpsz\" (UniqueName: \"kubernetes.io/projected/7450dd57-c0f7-47d6-ab28-108ea343d5fd-kube-api-access-9gpsz\") pod \"auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") " pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.687892 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.687803    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:23.823634 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:23.823600    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"]
Jul 26 16:16:23.826087 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:16:23.826058    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7450dd57_c0f7_47d6_ab28_108ea343d5fd.slice/crio-35fd5503521a473244fc7bb4bd6bae831f261b9a7c9a4dfa9468e0f31de0159a WatchSource:0}: Error finding container 35fd5503521a473244fc7bb4bd6bae831f261b9a7c9a4dfa9468e0f31de0159a: Status 404 returned error can't find the container with id 35fd5503521a473244fc7bb4bd6bae831f261b9a7c9a4dfa9468e0f31de0159a
Jul 26 16:16:24.159607 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:24.159574    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq" event={"ID":"7450dd57-c0f7-47d6-ab28-108ea343d5fd","Type":"ContainerStarted","Data":"d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35"}
Jul 26 16:16:24.159607 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:24.159613    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq" event={"ID":"7450dd57-c0f7-47d6-ab28-108ea343d5fd","Type":"ContainerStarted","Data":"35fd5503521a473244fc7bb4bd6bae831f261b9a7c9a4dfa9468e0f31de0159a"}
Jul 26 16:16:24.178320 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:24.178276    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq" podStartSLOduration=1.17826015 podStartE2EDuration="1.17826015s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:24.17733617 +0000 UTC m=+2283.361402309" watchObservedRunningTime="2026-07-26 16:16:24.17826015 +0000 UTC m=+2283.362326277"
Jul 26 16:16:28.691958 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:28.691923    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:16:33.688526 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:33.688487    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:33.688892 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:33.688575    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:33.701324 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:33.701303    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:34.221258 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:34.221211    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:16:40.373529 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.373497    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"]
Jul 26 16:16:40.377278 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.377259    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.380720 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.380683    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"auth-test-model-e0208c-kserve-self-signed-certs\""
Jul 26 16:16:40.380720 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.380691    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"default-dockercfg-x5pjt\""
Jul 26 16:16:40.380898 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.380691    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"openshift-service-ca.crt\""
Jul 26 16:16:40.380898 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.380748    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"kube-root-ca.crt\""
Jul 26 16:16:40.385887 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.385865    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"]
Jul 26 16:16:40.418609 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.418579    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tmp-dir\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.418776 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.418640    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-home\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.418776 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.418666    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-dshm\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.418776 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.418702    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-model-cache\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.418776 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.418752    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tls-certs\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.418952 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.418790    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkn2m\" (UniqueName: \"kubernetes.io/projected/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-kube-api-access-gkn2m\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.519556 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.519513    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tls-certs\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.519754 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.519567    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkn2m\" (UniqueName: \"kubernetes.io/projected/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-kube-api-access-gkn2m\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.519754 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.519645    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tmp-dir\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.519754 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.519706    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-home\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.519754 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.519728    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-dshm\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.519959 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.519792    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-model-cache\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.520061 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.520037    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tmp-dir\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.520132 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.520113    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-home\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.520210 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.520186    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-model-cache\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.521906 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.521888    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-dshm\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.522016 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.522000    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tls-certs\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.527990 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.527964    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkn2m\" (UniqueName: \"kubernetes.io/projected/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-kube-api-access-gkn2m\") pod \"auth-test-model-e0208c-kserve-64769477c9-j6gnt\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") " pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.688417 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.688318    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:40.818249 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:40.818204    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"]
Jul 26 16:16:40.820859 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:16:40.820826    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf21d9fcb_7e2b_46ed_b9e4_cd340a1107ef.slice/crio-7a6b16412b224f38e80546231b7bd9c66cbaa744e589e4dc20a990ffb81d16ef WatchSource:0}: Error finding container 7a6b16412b224f38e80546231b7bd9c66cbaa744e589e4dc20a990ffb81d16ef: Status 404 returned error can't find the container with id 7a6b16412b224f38e80546231b7bd9c66cbaa744e589e4dc20a990ffb81d16ef
Jul 26 16:16:41.245464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:41.245429    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt" event={"ID":"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef","Type":"ContainerStarted","Data":"f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692"}
Jul 26 16:16:41.245464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:41.245467    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt" event={"ID":"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef","Type":"ContainerStarted","Data":"7a6b16412b224f38e80546231b7bd9c66cbaa744e589e4dc20a990ffb81d16ef"}
Jul 26 16:16:41.262079 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:41.262026    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt" podStartSLOduration=1.262009898 podStartE2EDuration="1.262009898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:16:41.259982738 +0000 UTC m=+2300.444048881" watchObservedRunningTime="2026-07-26 16:16:41.262009898 +0000 UTC m=+2300.446076035"
Jul 26 16:16:44.989579 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:44.989538    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:16:50.689183 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:50.689148    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:50.689818 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:50.689199    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:50.701694 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:50.701667    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:16:51.306727 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:16:51.306696    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:17:11.281570 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:17:11.281533    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:17:31.794559 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:17:31.794521    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:17:50.586543 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:17:50.586505    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:18:11.886053 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:18:11.886018    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:18:29.886148 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:18:29.886109    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:18:52.990809 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:18:52.990736    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:19:10.991579 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:19:10.991544    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:19:33.985575 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:19:33.985538    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:19:45.298242 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:19:45.298196    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:20:02.383650 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:20:02.383614    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:20:24.887050 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:20:24.886954    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:20:43.099192 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:20:43.099161    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:21:10.082945 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:10.082906    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:21:15.072299 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:15.072258    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"]
Jul 26 16:21:15.883321 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:15.883290    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"]
Jul 26 16:21:21.434180 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:21.434148    2534 scope.go:117] "RemoveContainer" containerID="565dbb7f05de51b21b1aa12d241709ed247372267af6e82aadcb79e22c741203"
Jul 26 16:21:25.991157 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:25.991119    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:21:27.279427 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.279397    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"]
Jul 26 16:21:27.283890 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.283865    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.286399 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.286375    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aadfb2\"/\"sub-test-model-aadfb2-kserve-self-signed-certs\""
Jul 26 16:21:27.295120 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.295095    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"]
Jul 26 16:21:27.406958 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.406916    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-model-cache\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.407133 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.406970    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-home\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.407133 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.407018    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-tmp-dir\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.407206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.407128    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c402a54-b89c-4551-9eff-c75a0ded0722-tls-certs\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.407446 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.407312    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtmvm\" (UniqueName: \"kubernetes.io/projected/4c402a54-b89c-4551-9eff-c75a0ded0722-kube-api-access-mtmvm\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.407594 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.407457    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-dshm\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508277 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508221    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtmvm\" (UniqueName: \"kubernetes.io/projected/4c402a54-b89c-4551-9eff-c75a0ded0722-kube-api-access-mtmvm\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508277 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508281    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-dshm\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508509 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508315    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-model-cache\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508509 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508337    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-home\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508509 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508356    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-tmp-dir\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508509 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508404    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c402a54-b89c-4551-9eff-c75a0ded0722-tls-certs\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508773 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508747    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-home\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508840 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508773    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-model-cache\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.508878 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.508838    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-tmp-dir\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.510625 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.510600    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-dshm\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.510831 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.510813    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c402a54-b89c-4551-9eff-c75a0ded0722-tls-certs\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.517341 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.517321    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtmvm\" (UniqueName: \"kubernetes.io/projected/4c402a54-b89c-4551-9eff-c75a0ded0722-kube-api-access-mtmvm\") pod \"sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") " pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.594867 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.594809    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:27.735984 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:27.735956    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"]
Jul 26 16:21:27.738063 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:21:27.738032    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c402a54_b89c_4551_9eff_c75a0ded0722.slice/crio-b1ca447dcbe74a6a2d470c6cd9e116af8e56553434a09bfde47f8c2b84d98a18 WatchSource:0}: Error finding container b1ca447dcbe74a6a2d470c6cd9e116af8e56553434a09bfde47f8c2b84d98a18: Status 404 returned error can't find the container with id b1ca447dcbe74a6a2d470c6cd9e116af8e56553434a09bfde47f8c2b84d98a18
Jul 26 16:21:28.679692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:28.679650    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj" event={"ID":"4c402a54-b89c-4551-9eff-c75a0ded0722","Type":"ContainerStarted","Data":"d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07"}
Jul 26 16:21:28.679692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:28.679697    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj" event={"ID":"4c402a54-b89c-4551-9eff-c75a0ded0722","Type":"ContainerStarted","Data":"b1ca447dcbe74a6a2d470c6cd9e116af8e56553434a09bfde47f8c2b84d98a18"}
Jul 26 16:21:28.696467 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:28.696414    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj" podStartSLOduration=1.696395534 podStartE2EDuration="1.696395534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:21:28.695338641 +0000 UTC m=+2587.879404778" watchObservedRunningTime="2026-07-26 16:21:28.696395534 +0000 UTC m=+2587.880461672"
Jul 26 16:21:30.091131 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.091075    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq" podUID="7450dd57-c0f7-47d6-ab28-108ea343d5fd" containerName="main" containerID="cri-o://d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35" gracePeriod=45
Jul 26 16:21:30.338869 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.338846    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:21:30.434885 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.434809    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tls-certs\") pod \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") "
Jul 26 16:21:30.434885 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.434860    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-dshm\") pod \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") "
Jul 26 16:21:30.435102 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.434911    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gpsz\" (UniqueName: \"kubernetes.io/projected/7450dd57-c0f7-47d6-ab28-108ea343d5fd-kube-api-access-9gpsz\") pod \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") "
Jul 26 16:21:30.435102 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.434952    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-home\") pod \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") "
Jul 26 16:21:30.435102 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435008    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-model-cache\") pod \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") "
Jul 26 16:21:30.435102 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435035    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tmp-dir\") pod \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\" (UID: \"7450dd57-c0f7-47d6-ab28-108ea343d5fd\") "
Jul 26 16:21:30.435360 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435307    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7450dd57-c0f7-47d6-ab28-108ea343d5fd" (UID: "7450dd57-c0f7-47d6-ab28-108ea343d5fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:30.435360 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435323    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-home" (OuterVolumeSpecName: "home") pod "7450dd57-c0f7-47d6-ab28-108ea343d5fd" (UID: "7450dd57-c0f7-47d6-ab28-108ea343d5fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:30.435360 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435331    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-model-cache" (OuterVolumeSpecName: "model-cache") pod "7450dd57-c0f7-47d6-ab28-108ea343d5fd" (UID: "7450dd57-c0f7-47d6-ab28-108ea343d5fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:30.435502 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435460    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:30.435502 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435482    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:30.435502 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.435495    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:30.437063 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.437042    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-dshm" (OuterVolumeSpecName: "dshm") pod "7450dd57-c0f7-47d6-ab28-108ea343d5fd" (UID: "7450dd57-c0f7-47d6-ab28-108ea343d5fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:30.437148 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.437133    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7450dd57-c0f7-47d6-ab28-108ea343d5fd" (UID: "7450dd57-c0f7-47d6-ab28-108ea343d5fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:21:30.437194 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.437180    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7450dd57-c0f7-47d6-ab28-108ea343d5fd-kube-api-access-9gpsz" (OuterVolumeSpecName: "kube-api-access-9gpsz") pod "7450dd57-c0f7-47d6-ab28-108ea343d5fd" (UID: "7450dd57-c0f7-47d6-ab28-108ea343d5fd"). InnerVolumeSpecName "kube-api-access-9gpsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:21:30.536585 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.536556    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7450dd57-c0f7-47d6-ab28-108ea343d5fd-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:30.536585 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.536581    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gpsz\" (UniqueName: \"kubernetes.io/projected/7450dd57-c0f7-47d6-ab28-108ea343d5fd-kube-api-access-9gpsz\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:30.536751 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.536592    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7450dd57-c0f7-47d6-ab28-108ea343d5fd-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:30.690616 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.690525    2534 generic.go:356] "Generic (PLEG): container finished" podID="7450dd57-c0f7-47d6-ab28-108ea343d5fd" containerID="d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35" exitCode=0
Jul 26 16:21:30.690616 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.690574    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"
Jul 26 16:21:30.690838 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.690576    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq" event={"ID":"7450dd57-c0f7-47d6-ab28-108ea343d5fd","Type":"ContainerDied","Data":"d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35"}
Jul 26 16:21:30.690838 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.690694    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq" event={"ID":"7450dd57-c0f7-47d6-ab28-108ea343d5fd","Type":"ContainerDied","Data":"35fd5503521a473244fc7bb4bd6bae831f261b9a7c9a4dfa9468e0f31de0159a"}
Jul 26 16:21:30.690838 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.690717    2534 scope.go:117] "RemoveContainer" containerID="d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35"
Jul 26 16:21:30.699945 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.699929    2534 scope.go:117] "RemoveContainer" containerID="d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35"
Jul 26 16:21:30.700198 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:21:30.700182    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35\": container with ID starting with d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35 not found: ID does not exist" containerID="d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35"
Jul 26 16:21:30.700279 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.700210    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35"} err="failed to get container status \"d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35\": rpc error: code = NotFound desc = could not find container \"d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35\": container with ID starting with d7697c7e368e33ca05cce4330103f7472f237d4e998aa57b23eee6a40cc31e35 not found: ID does not exist"
Jul 26 16:21:30.712027 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.711923    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"]
Jul 26 16:21:30.714082 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.714063    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/auth-test-model-aadfb2-kserve-55fc49d4ff-vjxlq"]
Jul 26 16:21:30.903321 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:30.903286    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt" podUID="f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" containerName="main" containerID="cri-o://f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692" gracePeriod=45
Jul 26 16:21:31.154865 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.154842    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:21:31.243163 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243118    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tmp-dir\") pod \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") "
Jul 26 16:21:31.243372 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243211    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkn2m\" (UniqueName: \"kubernetes.io/projected/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-kube-api-access-gkn2m\") pod \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") "
Jul 26 16:21:31.243372 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243290    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tls-certs\") pod \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") "
Jul 26 16:21:31.243372 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243316    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-dshm\") pod \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") "
Jul 26 16:21:31.243372 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243366    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-home\") pod \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") "
Jul 26 16:21:31.243595 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243423    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-model-cache\") pod \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\" (UID: \"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef\") "
Jul 26 16:21:31.243595 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243457    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" (UID: "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:31.243692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243621    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-home" (OuterVolumeSpecName: "home") pod "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" (UID: "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:31.243692 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243674    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-model-cache" (OuterVolumeSpecName: "model-cache") pod "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" (UID: "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:31.243814 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243798    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:31.243853 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243817    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:31.243853 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.243841    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:31.245493 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.245471    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-dshm" (OuterVolumeSpecName: "dshm") pod "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" (UID: "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:21:31.245804 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.245786    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-kube-api-access-gkn2m" (OuterVolumeSpecName: "kube-api-access-gkn2m") pod "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" (UID: "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef"). InnerVolumeSpecName "kube-api-access-gkn2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:21:31.245804 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.245796    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" (UID: "f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:21:31.295261 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.295185    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7450dd57-c0f7-47d6-ab28-108ea343d5fd" path="/var/lib/kubelet/pods/7450dd57-c0f7-47d6-ab28-108ea343d5fd/volumes"
Jul 26 16:21:31.344291 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.344260    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkn2m\" (UniqueName: \"kubernetes.io/projected/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-kube-api-access-gkn2m\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:31.344291 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.344292    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:31.344291 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.344301    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:21:31.697759 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.697731    2534 generic.go:356] "Generic (PLEG): container finished" podID="f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" containerID="f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692" exitCode=0
Jul 26 16:21:31.697927 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.697770    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"
Jul 26 16:21:31.697927 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.697791    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt" event={"ID":"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef","Type":"ContainerDied","Data":"f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692"}
Jul 26 16:21:31.697927 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.697823    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt" event={"ID":"f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef","Type":"ContainerDied","Data":"7a6b16412b224f38e80546231b7bd9c66cbaa744e589e4dc20a990ffb81d16ef"}
Jul 26 16:21:31.697927 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.697840    2534 scope.go:117] "RemoveContainer" containerID="f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692"
Jul 26 16:21:31.706330 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.706314    2534 scope.go:117] "RemoveContainer" containerID="f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692"
Jul 26 16:21:31.706575 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:21:31.706558    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692\": container with ID starting with f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692 not found: ID does not exist" containerID="f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692"
Jul 26 16:21:31.706631 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.706584    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692"} err="failed to get container status \"f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692\": rpc error: code = NotFound desc = could not find container \"f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692\": container with ID starting with f81c00e5a256e0ee0af48a16583d916235f5e96e8d1b32425fcb1b8fe6fa0692 not found: ID does not exist"
Jul 26 16:21:31.713266 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.713245    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"]
Jul 26 16:21:31.715514 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:31.715496    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/auth-test-model-e0208c-kserve-64769477c9-j6gnt"]
Jul 26 16:21:33.295075 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:33.295041    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" path="/var/lib/kubelet/pods/f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef/volumes"
Jul 26 16:21:35.589732 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:35.589699    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:21:37.595496 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:37.595464    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:37.596118 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:37.596088    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:37.609857 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:37.609832    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:37.740987 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:37.740958    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:21:50.878145 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878072    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"]
Jul 26 16:21:50.878622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878458    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" containerName="main"
Jul 26 16:21:50.878622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878470    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" containerName="main"
Jul 26 16:21:50.878622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878480    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7450dd57-c0f7-47d6-ab28-108ea343d5fd" containerName="main"
Jul 26 16:21:50.878622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878485    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="7450dd57-c0f7-47d6-ab28-108ea343d5fd" containerName="main"
Jul 26 16:21:50.878622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878551    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="f21d9fcb-7e2b-46ed-b9e4-cd340a1107ef" containerName="main"
Jul 26 16:21:50.878622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.878562    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="7450dd57-c0f7-47d6-ab28-108ea343d5fd" containerName="main"
Jul 26 16:21:50.882860 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.882844    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:50.885599 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.885575    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"kube-root-ca.crt\""
Jul 26 16:21:50.885725 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.885581    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"sub-test-model-e0208c-kserve-self-signed-certs\""
Jul 26 16:21:50.885725 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.885640    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"openshift-service-ca.crt\""
Jul 26 16:21:50.885725 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.885660    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"default-dockercfg-x5pjt\""
Jul 26 16:21:50.890314 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:50.890290    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"]
Jul 26 16:21:51.019211 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.019177    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tls-certs\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.019421 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.019310    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62p97\" (UniqueName: \"kubernetes.io/projected/ebec527c-c7e7-44e6-bfe1-e8e78115939c-kube-api-access-62p97\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.019421 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.019372    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-dshm\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.019421 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.019394    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tmp-dir\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.019568 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.019433    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-home\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.019568 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.019463    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-model-cache\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120467 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120433    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-dshm\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120614 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120469    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tmp-dir\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120614 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120492    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-home\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120614 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120517    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-model-cache\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120614 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120586    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tls-certs\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120670    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62p97\" (UniqueName: \"kubernetes.io/projected/ebec527c-c7e7-44e6-bfe1-e8e78115939c-kube-api-access-62p97\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120798    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tmp-dir\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120924 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120858    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-home\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.120988 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.120963    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-model-cache\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.122751 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.122731    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-dshm\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.123131 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.123109    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tls-certs\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.129106 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.129063    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62p97\" (UniqueName: \"kubernetes.io/projected/ebec527c-c7e7-44e6-bfe1-e8e78115939c-kube-api-access-62p97\") pod \"sub-test-model-e0208c-kserve-6875885499-m6gtc\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") " pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.193029 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.193000    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:21:51.325644 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.325618    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"]
Jul 26 16:21:51.327199 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:21:51.327167    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebec527c_c7e7_44e6_bfe1_e8e78115939c.slice/crio-3473e4d125937bd21a0ac6a96366222beb3e39bfd9f6134f44c5c28b8f7b44d9 WatchSource:0}: Error finding container 3473e4d125937bd21a0ac6a96366222beb3e39bfd9f6134f44c5c28b8f7b44d9: Status 404 returned error can't find the container with id 3473e4d125937bd21a0ac6a96366222beb3e39bfd9f6134f44c5c28b8f7b44d9
Jul 26 16:21:51.801156 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.801119    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc" event={"ID":"ebec527c-c7e7-44e6-bfe1-e8e78115939c","Type":"ContainerStarted","Data":"ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1"}
Jul 26 16:21:51.801156 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.801162    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc" event={"ID":"ebec527c-c7e7-44e6-bfe1-e8e78115939c","Type":"ContainerStarted","Data":"3473e4d125937bd21a0ac6a96366222beb3e39bfd9f6134f44c5c28b8f7b44d9"}
Jul 26 16:21:51.821575 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:51.821530    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc" podStartSLOduration=1.821515523 podStartE2EDuration="1.821515523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:21:51.819903377 +0000 UTC m=+2611.003969515" watchObservedRunningTime="2026-07-26 16:21:51.821515523 +0000 UTC m=+2611.005581660"
Jul 26 16:21:54.887477 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:21:54.887443    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:22:01.193552 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:01.193506    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:22:01.193989 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:01.193589    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:22:01.206023 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:01.206000    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:22:01.868268 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:01.868221    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:22:21.452937 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:21.452905    2534 scope.go:117] "RemoveContainer" containerID="9ec5e8b45632eaa096c9a39bb805d5256718d99f671c72d43f6c83ddbbb085bf"
Jul 26 16:22:24.395083 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:24.395053    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:22:43.985405 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:43.985372    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:22:50.366173 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:50.366140    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"]
Jul 26 16:22:51.257964 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:51.257934    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"]
Jul 26 16:22:56.884572 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:56.884528    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"]
Jul 26 16:22:56.888201 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:56.888181    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:56.891538 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:56.891518    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aadfb2\"/\"rate-test-model-aadfb2-kserve-self-signed-certs\""
Jul 26 16:22:56.903932 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:56.903906    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"]
Jul 26 16:22:57.031488 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.031455    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c49dee97-e061-4c6f-bee0-57f769062c89-tls-certs\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.031664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.031493    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-dshm\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.031664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.031537    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-tmp-dir\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.031664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.031556    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-home\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.031664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.031590    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-model-cache\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.031664 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.031627    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zzw6\" (UniqueName: \"kubernetes.io/projected/c49dee97-e061-4c6f-bee0-57f769062c89-kube-api-access-4zzw6\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132303 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132271    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-model-cache\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132474 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132320    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zzw6\" (UniqueName: \"kubernetes.io/projected/c49dee97-e061-4c6f-bee0-57f769062c89-kube-api-access-4zzw6\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132474 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132395    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c49dee97-e061-4c6f-bee0-57f769062c89-tls-certs\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132474 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132414    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-dshm\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132474 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132445    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-tmp-dir\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132474 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132466    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-home\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132770 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132739    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-model-cache\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132894 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132749    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-tmp-dir\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.132894 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.132888    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-home\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.134823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.134762    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-dshm\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.134920 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.134859    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c49dee97-e061-4c6f-bee0-57f769062c89-tls-certs\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.141426 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.141405    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zzw6\" (UniqueName: \"kubernetes.io/projected/c49dee97-e061-4c6f-bee0-57f769062c89-kube-api-access-4zzw6\") pod \"rate-test-model-aadfb2-kserve-557d55b67-gzjb8\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") " pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.197154 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.197129    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:22:57.330995 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:57.330970    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"]
Jul 26 16:22:57.333890 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:22:57.333860    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc49dee97_e061_4c6f_bee0_57f769062c89.slice/crio-3ffe804f6f541b6f18a18aa135412ce86d5cd0bc66fc48c49668005640324991 WatchSource:0}: Error finding container 3ffe804f6f541b6f18a18aa135412ce86d5cd0bc66fc48c49668005640324991: Status 404 returned error can't find the container with id 3ffe804f6f541b6f18a18aa135412ce86d5cd0bc66fc48c49668005640324991
Jul 26 16:22:58.135969 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:58.135932    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8" event={"ID":"c49dee97-e061-4c6f-bee0-57f769062c89","Type":"ContainerStarted","Data":"f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8"}
Jul 26 16:22:58.135969 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:58.135973    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8" event={"ID":"c49dee97-e061-4c6f-bee0-57f769062c89","Type":"ContainerStarted","Data":"3ffe804f6f541b6f18a18aa135412ce86d5cd0bc66fc48c49668005640324991"}
Jul 26 16:22:58.158283 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:22:58.158210    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8" podStartSLOduration=2.158195465 podStartE2EDuration="2.158195465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:22:58.156071387 +0000 UTC m=+2677.340137537" watchObservedRunningTime="2026-07-26 16:22:58.158195465 +0000 UTC m=+2677.342261580"
Jul 26 16:23:01.414989 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:01.414952    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:23:05.384487 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.384441    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj" podUID="4c402a54-b89c-4551-9eff-c75a0ded0722" containerName="main" containerID="cri-o://d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07" gracePeriod=45
Jul 26 16:23:05.639694 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.639634    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:23:05.711264 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711218    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c402a54-b89c-4551-9eff-c75a0ded0722-tls-certs\") pod \"4c402a54-b89c-4551-9eff-c75a0ded0722\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") "
Jul 26 16:23:05.711404 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711279    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-dshm\") pod \"4c402a54-b89c-4551-9eff-c75a0ded0722\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") "
Jul 26 16:23:05.711404 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711317    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtmvm\" (UniqueName: \"kubernetes.io/projected/4c402a54-b89c-4551-9eff-c75a0ded0722-kube-api-access-mtmvm\") pod \"4c402a54-b89c-4551-9eff-c75a0ded0722\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") "
Jul 26 16:23:05.711404 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711341    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-tmp-dir\") pod \"4c402a54-b89c-4551-9eff-c75a0ded0722\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") "
Jul 26 16:23:05.711404 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711367    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-home\") pod \"4c402a54-b89c-4551-9eff-c75a0ded0722\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") "
Jul 26 16:23:05.711606 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711457    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-model-cache\") pod \"4c402a54-b89c-4551-9eff-c75a0ded0722\" (UID: \"4c402a54-b89c-4551-9eff-c75a0ded0722\") "
Jul 26 16:23:05.711654 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711628    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4c402a54-b89c-4551-9eff-c75a0ded0722" (UID: "4c402a54-b89c-4551-9eff-c75a0ded0722"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:05.711654 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711636    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-home" (OuterVolumeSpecName: "home") pod "4c402a54-b89c-4551-9eff-c75a0ded0722" (UID: "4c402a54-b89c-4551-9eff-c75a0ded0722"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:05.711761 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711719    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-model-cache" (OuterVolumeSpecName: "model-cache") pod "4c402a54-b89c-4551-9eff-c75a0ded0722" (UID: "4c402a54-b89c-4551-9eff-c75a0ded0722"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:05.711961 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711940    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:05.712017 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711965    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:05.712017 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.711978    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:05.713383 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.713361    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c402a54-b89c-4551-9eff-c75a0ded0722-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4c402a54-b89c-4551-9eff-c75a0ded0722" (UID: "4c402a54-b89c-4551-9eff-c75a0ded0722"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:23:05.713472 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.713383    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-dshm" (OuterVolumeSpecName: "dshm") pod "4c402a54-b89c-4551-9eff-c75a0ded0722" (UID: "4c402a54-b89c-4551-9eff-c75a0ded0722"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:05.713516 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.713466    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c402a54-b89c-4551-9eff-c75a0ded0722-kube-api-access-mtmvm" (OuterVolumeSpecName: "kube-api-access-mtmvm") pod "4c402a54-b89c-4551-9eff-c75a0ded0722" (UID: "4c402a54-b89c-4551-9eff-c75a0ded0722"). InnerVolumeSpecName "kube-api-access-mtmvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:23:05.813309 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.813259    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c402a54-b89c-4551-9eff-c75a0ded0722-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:05.813309 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.813303    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtmvm\" (UniqueName: \"kubernetes.io/projected/4c402a54-b89c-4551-9eff-c75a0ded0722-kube-api-access-mtmvm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:05.813309 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:05.813314    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c402a54-b89c-4551-9eff-c75a0ded0722-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:06.175297 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.175262    2534 generic.go:356] "Generic (PLEG): container finished" podID="4c402a54-b89c-4551-9eff-c75a0ded0722" containerID="d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07" exitCode=0
Jul 26 16:23:06.175469 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.175310    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"
Jul 26 16:23:06.175469 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.175326    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj" event={"ID":"4c402a54-b89c-4551-9eff-c75a0ded0722","Type":"ContainerDied","Data":"d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07"}
Jul 26 16:23:06.175469 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.175362    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj" event={"ID":"4c402a54-b89c-4551-9eff-c75a0ded0722","Type":"ContainerDied","Data":"b1ca447dcbe74a6a2d470c6cd9e116af8e56553434a09bfde47f8c2b84d98a18"}
Jul 26 16:23:06.175469 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.175385    2534 scope.go:117] "RemoveContainer" containerID="d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07"
Jul 26 16:23:06.184502 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.184480    2534 scope.go:117] "RemoveContainer" containerID="d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07"
Jul 26 16:23:06.184779 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:23:06.184761    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07\": container with ID starting with d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07 not found: ID does not exist" containerID="d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07"
Jul 26 16:23:06.184832 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.184788    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07"} err="failed to get container status \"d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07\": rpc error: code = NotFound desc = could not find container \"d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07\": container with ID starting with d570977a75eec0a15b9fe0501b7311eb0decab80388cc1e398a59c635c66eb07 not found: ID does not exist"
Jul 26 16:23:06.201006 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.200981    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"]
Jul 26 16:23:06.204176 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.204156    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/sub-test-model-aadfb2-kserve-6f6d5c8c6c-ln6pj"]
Jul 26 16:23:06.277121 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.277095    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc" podUID="ebec527c-c7e7-44e6-bfe1-e8e78115939c" containerName="main" containerID="cri-o://ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1" gracePeriod=45
Jul 26 16:23:06.525304 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.525280    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:23:06.620288 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620254    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-home\") pod \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") "
Jul 26 16:23:06.620468 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620315    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tls-certs\") pod \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") "
Jul 26 16:23:06.620468 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620351    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-model-cache\") pod \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") "
Jul 26 16:23:06.620468 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620385    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62p97\" (UniqueName: \"kubernetes.io/projected/ebec527c-c7e7-44e6-bfe1-e8e78115939c-kube-api-access-62p97\") pod \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") "
Jul 26 16:23:06.620468 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620403    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-dshm\") pod \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") "
Jul 26 16:23:06.620468 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620426    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tmp-dir\") pod \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\" (UID: \"ebec527c-c7e7-44e6-bfe1-e8e78115939c\") "
Jul 26 16:23:06.620721 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620576    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-home" (OuterVolumeSpecName: "home") pod "ebec527c-c7e7-44e6-bfe1-e8e78115939c" (UID: "ebec527c-c7e7-44e6-bfe1-e8e78115939c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:06.620721 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620668    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ebec527c-c7e7-44e6-bfe1-e8e78115939c" (UID: "ebec527c-c7e7-44e6-bfe1-e8e78115939c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:06.620818 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620796    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ebec527c-c7e7-44e6-bfe1-e8e78115939c" (UID: "ebec527c-c7e7-44e6-bfe1-e8e78115939c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:06.620950 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620934    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:06.621018 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620954    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:06.621018 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.620966    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:06.622574 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.622545    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebec527c-c7e7-44e6-bfe1-e8e78115939c-kube-api-access-62p97" (OuterVolumeSpecName: "kube-api-access-62p97") pod "ebec527c-c7e7-44e6-bfe1-e8e78115939c" (UID: "ebec527c-c7e7-44e6-bfe1-e8e78115939c"). InnerVolumeSpecName "kube-api-access-62p97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:23:06.622574 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.622555    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-dshm" (OuterVolumeSpecName: "dshm") pod "ebec527c-c7e7-44e6-bfe1-e8e78115939c" (UID: "ebec527c-c7e7-44e6-bfe1-e8e78115939c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:23:06.622714 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.622590    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ebec527c-c7e7-44e6-bfe1-e8e78115939c" (UID: "ebec527c-c7e7-44e6-bfe1-e8e78115939c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:23:06.721644 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.721614    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62p97\" (UniqueName: \"kubernetes.io/projected/ebec527c-c7e7-44e6-bfe1-e8e78115939c-kube-api-access-62p97\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:06.721644 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.721642    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebec527c-c7e7-44e6-bfe1-e8e78115939c-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:06.721835 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:06.721656    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebec527c-c7e7-44e6-bfe1-e8e78115939c-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:23:07.181558 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.181480    2534 generic.go:356] "Generic (PLEG): container finished" podID="ebec527c-c7e7-44e6-bfe1-e8e78115939c" containerID="ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1" exitCode=0
Jul 26 16:23:07.181558 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.181521    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"
Jul 26 16:23:07.181806 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.181565    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc" event={"ID":"ebec527c-c7e7-44e6-bfe1-e8e78115939c","Type":"ContainerDied","Data":"ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1"}
Jul 26 16:23:07.181806 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.181608    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc" event={"ID":"ebec527c-c7e7-44e6-bfe1-e8e78115939c","Type":"ContainerDied","Data":"3473e4d125937bd21a0ac6a96366222beb3e39bfd9f6134f44c5c28b8f7b44d9"}
Jul 26 16:23:07.181806 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.181635    2534 scope.go:117] "RemoveContainer" containerID="ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1"
Jul 26 16:23:07.190603 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.190585    2534 scope.go:117] "RemoveContainer" containerID="ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1"
Jul 26 16:23:07.190849 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:23:07.190830    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1\": container with ID starting with ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1 not found: ID does not exist" containerID="ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1"
Jul 26 16:23:07.190919 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.190861    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1"} err="failed to get container status \"ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1\": rpc error: code = NotFound desc = could not find container \"ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1\": container with ID starting with ca387537d43105927becdc354401f6f2012b7d8ad04e2db884fc513821571bb1 not found: ID does not exist"
Jul 26 16:23:07.197858 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.197833    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:23:07.197937 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.197878    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:23:07.205985 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.205964    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"]
Jul 26 16:23:07.214802 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.214780    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:23:07.214939 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.214926    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/sub-test-model-e0208c-kserve-6875885499-m6gtc"]
Jul 26 16:23:07.293660 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.293628    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c402a54-b89c-4551-9eff-c75a0ded0722" path="/var/lib/kubelet/pods/4c402a54-b89c-4551-9eff-c75a0ded0722/volumes"
Jul 26 16:23:07.294154 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:07.294138    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebec527c-c7e7-44e6-bfe1-e8e78115939c" path="/var/lib/kubelet/pods/ebec527c-c7e7-44e6-bfe1-e8e78115939c/volumes"
Jul 26 16:23:08.199248 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:08.199208    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:23:31.384081 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384050    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"]
Jul 26 16:23:31.384541 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384470    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebec527c-c7e7-44e6-bfe1-e8e78115939c" containerName="main"
Jul 26 16:23:31.384541 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384484    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebec527c-c7e7-44e6-bfe1-e8e78115939c" containerName="main"
Jul 26 16:23:31.384541 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384511    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c402a54-b89c-4551-9eff-c75a0ded0722" containerName="main"
Jul 26 16:23:31.384541 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384517    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c402a54-b89c-4551-9eff-c75a0ded0722" containerName="main"
Jul 26 16:23:31.384677 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384579    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c402a54-b89c-4551-9eff-c75a0ded0722" containerName="main"
Jul 26 16:23:31.384677 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.384589    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebec527c-c7e7-44e6-bfe1-e8e78115939c" containerName="main"
Jul 26 16:23:31.387724 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.387701    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.390456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.390430    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"default-dockercfg-x5pjt\""
Jul 26 16:23:31.391587 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.391560    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"kube-root-ca.crt\""
Jul 26 16:23:31.391698 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.391566    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"rate-test-model-e0208c-kserve-self-signed-certs\""
Jul 26 16:23:31.391698 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.391583    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e0208c\"/\"openshift-service-ca.crt\""
Jul 26 16:23:31.402031 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.402009    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"]
Jul 26 16:23:31.442451 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.442420    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-home\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.442577 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.442455    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-tmp-dir\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.442577 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.442524    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f2bead-0939-46ca-9b35-a7ee1f726c28-tls-certs\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.442683 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.442607    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-658z8\" (UniqueName: \"kubernetes.io/projected/20f2bead-0939-46ca-9b35-a7ee1f726c28-kube-api-access-658z8\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.442683 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.442644    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-model-cache\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.442683 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.442668    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-dshm\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543372 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543332    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f2bead-0939-46ca-9b35-a7ee1f726c28-tls-certs\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543542 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543405    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-658z8\" (UniqueName: \"kubernetes.io/projected/20f2bead-0939-46ca-9b35-a7ee1f726c28-kube-api-access-658z8\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543542 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543436    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-model-cache\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543542 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543464    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-dshm\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543542 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543517    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-home\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543542 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543538    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-tmp-dir\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543885 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543850    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-model-cache\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.543885 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543873    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-home\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.544031 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.543964    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-tmp-dir\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.545719 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.545699    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-dshm\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.545981 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.545966    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f2bead-0939-46ca-9b35-a7ee1f726c28-tls-certs\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.552778 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.552758    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-658z8\" (UniqueName: \"kubernetes.io/projected/20f2bead-0939-46ca-9b35-a7ee1f726c28-kube-api-access-658z8\") pod \"rate-test-model-e0208c-kserve-7858d94d8c-p7lr7\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") " pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.698751 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.698720    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:31.849038 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:31.849003    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"]
Jul 26 16:23:31.850452 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:23:31.850426    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20f2bead_0939_46ca_9b35_a7ee1f726c28.slice/crio-6fcd97a06fce2a88b9f3f88ec4878404d8a2991326183a060b5f609c3a6e477a WatchSource:0}: Error finding container 6fcd97a06fce2a88b9f3f88ec4878404d8a2991326183a060b5f609c3a6e477a: Status 404 returned error can't find the container with id 6fcd97a06fce2a88b9f3f88ec4878404d8a2991326183a060b5f609c3a6e477a
Jul 26 16:23:32.316032 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:32.315996    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7" event={"ID":"20f2bead-0939-46ca-9b35-a7ee1f726c28","Type":"ContainerStarted","Data":"7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d"}
Jul 26 16:23:32.316200 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:32.316042    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7" event={"ID":"20f2bead-0939-46ca-9b35-a7ee1f726c28","Type":"ContainerStarted","Data":"6fcd97a06fce2a88b9f3f88ec4878404d8a2991326183a060b5f609c3a6e477a"}
Jul 26 16:23:32.336499 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:32.336454    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7" podStartSLOduration=1.336440369 podStartE2EDuration="1.336440369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:23:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:23:32.335448712 +0000 UTC m=+2711.519514849" watchObservedRunningTime="2026-07-26 16:23:32.336440369 +0000 UTC m=+2711.520506586"
Jul 26 16:23:35.212741 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:35.212708    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:23:41.699641 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:41.699601    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:41.700005 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:41.699659    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:41.712004 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:41.711974    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:23:42.373302 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:23:42.373272    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:24:04.793351 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:04.793314    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:24:18.611703 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:18.611668    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:24:33.682462 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:33.682424    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:24:37.324811 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:37.324778    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"]
Jul 26 16:24:38.227490 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:38.227458    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"]
Jul 26 16:24:50.286725 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:50.286647    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:24:52.343293 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.343207    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8" podUID="c49dee97-e061-4c6f-bee0-57f769062c89" containerName="main" containerID="cri-o://f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8" gracePeriod=45
Jul 26 16:24:52.587085 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.587063    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:24:52.684804 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.684732    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c49dee97-e061-4c6f-bee0-57f769062c89-tls-certs\") pod \"c49dee97-e061-4c6f-bee0-57f769062c89\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") "
Jul 26 16:24:52.684804 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.684801    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-model-cache\") pod \"c49dee97-e061-4c6f-bee0-57f769062c89\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") "
Jul 26 16:24:52.685007 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.684836    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-dshm\") pod \"c49dee97-e061-4c6f-bee0-57f769062c89\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") "
Jul 26 16:24:52.685007 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.684859    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-tmp-dir\") pod \"c49dee97-e061-4c6f-bee0-57f769062c89\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") "
Jul 26 16:24:52.685007 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.684906    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zzw6\" (UniqueName: \"kubernetes.io/projected/c49dee97-e061-4c6f-bee0-57f769062c89-kube-api-access-4zzw6\") pod \"c49dee97-e061-4c6f-bee0-57f769062c89\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") "
Jul 26 16:24:52.685007 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.684947    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-home\") pod \"c49dee97-e061-4c6f-bee0-57f769062c89\" (UID: \"c49dee97-e061-4c6f-bee0-57f769062c89\") "
Jul 26 16:24:52.685209 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.685070    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-model-cache" (OuterVolumeSpecName: "model-cache") pod "c49dee97-e061-4c6f-bee0-57f769062c89" (UID: "c49dee97-e061-4c6f-bee0-57f769062c89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:52.685209 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.685137    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c49dee97-e061-4c6f-bee0-57f769062c89" (UID: "c49dee97-e061-4c6f-bee0-57f769062c89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:52.685356 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.685276    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-home" (OuterVolumeSpecName: "home") pod "c49dee97-e061-4c6f-bee0-57f769062c89" (UID: "c49dee97-e061-4c6f-bee0-57f769062c89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:52.685356 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.685312    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:52.685356 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.685334    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:52.686920 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.686889    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-dshm" (OuterVolumeSpecName: "dshm") pod "c49dee97-e061-4c6f-bee0-57f769062c89" (UID: "c49dee97-e061-4c6f-bee0-57f769062c89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:52.687044 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.686954    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c49dee97-e061-4c6f-bee0-57f769062c89-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c49dee97-e061-4c6f-bee0-57f769062c89" (UID: "c49dee97-e061-4c6f-bee0-57f769062c89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:24:52.687044 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.686958    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c49dee97-e061-4c6f-bee0-57f769062c89-kube-api-access-4zzw6" (OuterVolumeSpecName: "kube-api-access-4zzw6") pod "c49dee97-e061-4c6f-bee0-57f769062c89" (UID: "c49dee97-e061-4c6f-bee0-57f769062c89"). InnerVolumeSpecName "kube-api-access-4zzw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:24:52.706735 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.706714    2534 generic.go:356] "Generic (PLEG): container finished" podID="c49dee97-e061-4c6f-bee0-57f769062c89" containerID="f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8" exitCode=0
Jul 26 16:24:52.706828 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.706752    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"
Jul 26 16:24:52.706828 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.706784    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8" event={"ID":"c49dee97-e061-4c6f-bee0-57f769062c89","Type":"ContainerDied","Data":"f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8"}
Jul 26 16:24:52.706828 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.706813    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8" event={"ID":"c49dee97-e061-4c6f-bee0-57f769062c89","Type":"ContainerDied","Data":"3ffe804f6f541b6f18a18aa135412ce86d5cd0bc66fc48c49668005640324991"}
Jul 26 16:24:52.706950 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.706830    2534 scope.go:117] "RemoveContainer" containerID="f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8"
Jul 26 16:24:52.716086 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.716070    2534 scope.go:117] "RemoveContainer" containerID="f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8"
Jul 26 16:24:52.716363 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:24:52.716343    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8\": container with ID starting with f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8 not found: ID does not exist" containerID="f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8"
Jul 26 16:24:52.716424 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.716371    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8"} err="failed to get container status \"f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8\": rpc error: code = NotFound desc = could not find container \"f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8\": container with ID starting with f410d220b4203eef6a755618b97c96f897d457aea74c03afed17e024623b0ec8 not found: ID does not exist"
Jul 26 16:24:52.729344 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.729323    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"]
Jul 26 16:24:52.732917 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.732897    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aadfb2/rate-test-model-aadfb2-kserve-557d55b67-gzjb8"]
Jul 26 16:24:52.786530 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.786498    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c49dee97-e061-4c6f-bee0-57f769062c89-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:52.786530 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.786526    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:52.786686 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.786539    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zzw6\" (UniqueName: \"kubernetes.io/projected/c49dee97-e061-4c6f-bee0-57f769062c89-kube-api-access-4zzw6\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:52.786686 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:52.786552    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c49dee97-e061-4c6f-bee0-57f769062c89-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.246008 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.245973    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7" podUID="20f2bead-0939-46ca-9b35-a7ee1f726c28" containerName="main" containerID="cri-o://7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d" gracePeriod=45
Jul 26 16:24:53.295124 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.295085    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c49dee97-e061-4c6f-bee0-57f769062c89" path="/var/lib/kubelet/pods/c49dee97-e061-4c6f-bee0-57f769062c89/volumes"
Jul 26 16:24:53.497679 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.497621    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:24:53.593630 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593599    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-home\") pod \"20f2bead-0939-46ca-9b35-a7ee1f726c28\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") "
Jul 26 16:24:53.593782 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593640    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f2bead-0939-46ca-9b35-a7ee1f726c28-tls-certs\") pod \"20f2bead-0939-46ca-9b35-a7ee1f726c28\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") "
Jul 26 16:24:53.593782 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593679    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-model-cache\") pod \"20f2bead-0939-46ca-9b35-a7ee1f726c28\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") "
Jul 26 16:24:53.593782 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593695    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-tmp-dir\") pod \"20f2bead-0939-46ca-9b35-a7ee1f726c28\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") "
Jul 26 16:24:53.593782 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593716    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-658z8\" (UniqueName: \"kubernetes.io/projected/20f2bead-0939-46ca-9b35-a7ee1f726c28-kube-api-access-658z8\") pod \"20f2bead-0939-46ca-9b35-a7ee1f726c28\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") "
Jul 26 16:24:53.593960 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593813    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-dshm\") pod \"20f2bead-0939-46ca-9b35-a7ee1f726c28\" (UID: \"20f2bead-0939-46ca-9b35-a7ee1f726c28\") "
Jul 26 16:24:53.593960 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593862    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-home" (OuterVolumeSpecName: "home") pod "20f2bead-0939-46ca-9b35-a7ee1f726c28" (UID: "20f2bead-0939-46ca-9b35-a7ee1f726c28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:53.594065 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593955    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-model-cache" (OuterVolumeSpecName: "model-cache") pod "20f2bead-0939-46ca-9b35-a7ee1f726c28" (UID: "20f2bead-0939-46ca-9b35-a7ee1f726c28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:53.594065 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.593968    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "20f2bead-0939-46ca-9b35-a7ee1f726c28" (UID: "20f2bead-0939-46ca-9b35-a7ee1f726c28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:53.594176 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.594096    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.594176 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.594108    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.594176 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.594121    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.595786 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.595761    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f2bead-0939-46ca-9b35-a7ee1f726c28-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "20f2bead-0939-46ca-9b35-a7ee1f726c28" (UID: "20f2bead-0939-46ca-9b35-a7ee1f726c28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:24:53.595896 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.595796    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-dshm" (OuterVolumeSpecName: "dshm") pod "20f2bead-0939-46ca-9b35-a7ee1f726c28" (UID: "20f2bead-0939-46ca-9b35-a7ee1f726c28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:24:53.595896 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.595813    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f2bead-0939-46ca-9b35-a7ee1f726c28-kube-api-access-658z8" (OuterVolumeSpecName: "kube-api-access-658z8") pod "20f2bead-0939-46ca-9b35-a7ee1f726c28" (UID: "20f2bead-0939-46ca-9b35-a7ee1f726c28"). InnerVolumeSpecName "kube-api-access-658z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:24:53.695198 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.695176    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20f2bead-0939-46ca-9b35-a7ee1f726c28-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.695198 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.695195    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20f2bead-0939-46ca-9b35-a7ee1f726c28-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.695349 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.695205    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-658z8\" (UniqueName: \"kubernetes.io/projected/20f2bead-0939-46ca-9b35-a7ee1f726c28-kube-api-access-658z8\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:53.712615 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.712590    2534 generic.go:356] "Generic (PLEG): container finished" podID="20f2bead-0939-46ca-9b35-a7ee1f726c28" containerID="7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d" exitCode=0
Jul 26 16:24:53.712718 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.712632    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"
Jul 26 16:24:53.712718 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.712682    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7" event={"ID":"20f2bead-0939-46ca-9b35-a7ee1f726c28","Type":"ContainerDied","Data":"7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d"}
Jul 26 16:24:53.712801 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.712727    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7" event={"ID":"20f2bead-0939-46ca-9b35-a7ee1f726c28","Type":"ContainerDied","Data":"6fcd97a06fce2a88b9f3f88ec4878404d8a2991326183a060b5f609c3a6e477a"}
Jul 26 16:24:53.712801 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.712751    2534 scope.go:117] "RemoveContainer" containerID="7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d"
Jul 26 16:24:53.721464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.721449    2534 scope.go:117] "RemoveContainer" containerID="7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d"
Jul 26 16:24:53.721701 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:24:53.721681    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d\": container with ID starting with 7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d not found: ID does not exist" containerID="7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d"
Jul 26 16:24:53.721750 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.721707    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d"} err="failed to get container status \"7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d\": rpc error: code = NotFound desc = could not find container \"7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d\": container with ID starting with 7d708a776baac207d2cae55c10e782830b0059261771eec152acb49938711b0d not found: ID does not exist"
Jul 26 16:24:53.733391 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.733369    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"]
Jul 26 16:24:53.738342 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.738323    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e0208c/rate-test-model-e0208c-kserve-7858d94d8c-p7lr7"]
Jul 26 16:24:53.906521 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.906437    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7633-140497989957216"]
Jul 26 16:24:53.906916 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.906898    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c49dee97-e061-4c6f-bee0-57f769062c89" containerName="main"
Jul 26 16:24:53.907008 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.906919    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c49dee97-e061-4c6f-bee0-57f769062c89" containerName="main"
Jul 26 16:24:53.907008 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.906932    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20f2bead-0939-46ca-9b35-a7ee1f726c28" containerName="main"
Jul 26 16:24:53.907008 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.906958    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="20f2bead-0939-46ca-9b35-a7ee1f726c28" containerName="main"
Jul 26 16:24:53.907173 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.907045    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c49dee97-e061-4c6f-bee0-57f769062c89" containerName="main"
Jul 26 16:24:53.907173 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.907059    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="20f2bead-0939-46ca-9b35-a7ee1f726c28" containerName="main"
Jul 26 16:24:53.911440 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.911418    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:24:53.913868 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.913843    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-p7n2s\""
Jul 26 16:24:53.914571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.914550    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7633-140497989957216"]
Jul 26 16:24:53.998724 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:53.998697    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gggrn\" (UniqueName: \"kubernetes.io/projected/48982b4b-6d93-48fa-ae7e-3c7766725464-kube-api-access-gggrn\") pod \"test-curl-7633-140497989957216\" (UID: \"48982b4b-6d93-48fa-ae7e-3c7766725464\") " pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:24:54.099288 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:54.099265    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gggrn\" (UniqueName: \"kubernetes.io/projected/48982b4b-6d93-48fa-ae7e-3c7766725464-kube-api-access-gggrn\") pod \"test-curl-7633-140497989957216\" (UID: \"48982b4b-6d93-48fa-ae7e-3c7766725464\") " pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:24:54.107591 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:54.107566    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gggrn\" (UniqueName: \"kubernetes.io/projected/48982b4b-6d93-48fa-ae7e-3c7766725464-kube-api-access-gggrn\") pod \"test-curl-7633-140497989957216\" (UID: \"48982b4b-6d93-48fa-ae7e-3c7766725464\") " pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:24:54.221540 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:54.221516    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:24:54.344907 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:54.344879    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7633-140497989957216"]
Jul 26 16:24:54.347272 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:24:54.347245    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48982b4b_6d93_48fa_ae7e_3c7766725464.slice/crio-ec71e06e46946fe70137625c977e7704dab2f185552da1481432d6c3ffee6999 WatchSource:0}: Error finding container ec71e06e46946fe70137625c977e7704dab2f185552da1481432d6c3ffee6999: Status 404 returned error can't find the container with id ec71e06e46946fe70137625c977e7704dab2f185552da1481432d6c3ffee6999
Jul 26 16:24:54.348501 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:54.348485    2534 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 16:24:54.721901 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:54.721862    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7633-140497989957216" event={"ID":"48982b4b-6d93-48fa-ae7e-3c7766725464","Type":"ContainerStarted","Data":"ec71e06e46946fe70137625c977e7704dab2f185552da1481432d6c3ffee6999"}
Jul 26 16:24:55.299677 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:55.299598    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20f2bead-0939-46ca-9b35-a7ee1f726c28" path="/var/lib/kubelet/pods/20f2bead-0939-46ca-9b35-a7ee1f726c28/volumes"
Jul 26 16:24:55.729848 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:55.729807    2534 generic.go:356] "Generic (PLEG): container finished" podID="48982b4b-6d93-48fa-ae7e-3c7766725464" containerID="a703de8b2de33e8101a66c151a631eda78597464b4419783b3f7737f2a03224e" exitCode=0
Jul 26 16:24:55.730197 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:55.729895    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7633-140497989957216" event={"ID":"48982b4b-6d93-48fa-ae7e-3c7766725464","Type":"ContainerDied","Data":"a703de8b2de33e8101a66c151a631eda78597464b4419783b3f7737f2a03224e"}
Jul 26 16:24:56.868534 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:56.868512    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:24:56.906663 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:56.906635    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7633-140497989957216_48982b4b-6d93-48fa-ae7e-3c7766725464/test-curl-7633-140497989957216/0.log"
Jul 26 16:24:56.924074 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:56.924053    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gggrn\" (UniqueName: \"kubernetes.io/projected/48982b4b-6d93-48fa-ae7e-3c7766725464-kube-api-access-gggrn\") pod \"48982b4b-6d93-48fa-ae7e-3c7766725464\" (UID: \"48982b4b-6d93-48fa-ae7e-3c7766725464\") "
Jul 26 16:24:56.926167 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:56.926141    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48982b4b-6d93-48fa-ae7e-3c7766725464-kube-api-access-gggrn" (OuterVolumeSpecName: "kube-api-access-gggrn") pod "48982b4b-6d93-48fa-ae7e-3c7766725464" (UID: "48982b4b-6d93-48fa-ae7e-3c7766725464"). InnerVolumeSpecName "kube-api-access-gggrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:24:56.929242 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:56.929205    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7633-140497989957216"]
Jul 26 16:24:56.933221 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:56.933201    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7633-140497989957216"]
Jul 26 16:24:57.025404 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:57.025350    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gggrn\" (UniqueName: \"kubernetes.io/projected/48982b4b-6d93-48fa-ae7e-3c7766725464-kube-api-access-gggrn\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:24:57.294136 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:57.294065    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48982b4b-6d93-48fa-ae7e-3c7766725464" path="/var/lib/kubelet/pods/48982b4b-6d93-48fa-ae7e-3c7766725464/volumes"
Jul 26 16:24:57.741255 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:57.741209    2534 scope.go:117] "RemoveContainer" containerID="a703de8b2de33e8101a66c151a631eda78597464b4419783b3f7737f2a03224e"
Jul 26 16:24:57.741434 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:24:57.741209    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7633-140497989957216"
Jul 26 16:25:06.489542 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:06.489509    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:25:16.224961 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.224928    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"]
Jul 26 16:25:16.225456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.225318    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48982b4b-6d93-48fa-ae7e-3c7766725464" containerName="test-curl-7633-140497989957216"
Jul 26 16:25:16.225456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.225329    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="48982b4b-6d93-48fa-ae7e-3c7766725464" containerName="test-curl-7633-140497989957216"
Jul 26 16:25:16.225456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.225413    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="48982b4b-6d93-48fa-ae7e-3c7766725464" containerName="test-curl-7633-140497989957216"
Jul 26 16:25:16.228522 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.228502    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.231012 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.230994    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-8c3dba-openshift-default-dockercfg-bf7rw\""
Jul 26 16:25:16.231122 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.231104    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 26 16:25:16.244116 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.244092    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"]
Jul 26 16:25:16.296314 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296272    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-certs\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296481 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296394    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea1f90-cf56-416b-bd20-79301599a9ea-istiod-ca-cert\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296481 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296424    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-envoy\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296481 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296442    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-credential-socket\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296481 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296464    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-data\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296481 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296480    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-podinfo\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296678 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296576    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9kvq\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-kube-api-access-f9kvq\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296678 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296629    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-socket\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.296757 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.296686    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-token\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.397784 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.397746    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea1f90-cf56-416b-bd20-79301599a9ea-istiod-ca-cert\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.397943 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.397874    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-envoy\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.397943 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.397908    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-credential-socket\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398046 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398026    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-data\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398084 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398057    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-podinfo\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398123 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398104    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9kvq\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-kube-api-access-f9kvq\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398171 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398155    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-socket\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398226 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398209    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-token\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398305 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398222    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-credential-socket\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398365 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398301    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-certs\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398582 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398562    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea1f90-cf56-416b-bd20-79301599a9ea-istiod-ca-cert\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398846 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398818    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-certs\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.398922 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.398822    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-socket\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.399068 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.399047    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-data\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.400120 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.400097    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-envoy\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.400480 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.400463    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-podinfo\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.413821 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.413798    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-token\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.414397 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.414379    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9kvq\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-kube-api-access-f9kvq\") pod \"e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") " pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.539152 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.539076    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:16.673919 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.673894    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"]
Jul 26 16:25:16.676106 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:25:16.676078    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8ea1f90_cf56_416b_bd20_79301599a9ea.slice/crio-3ae18e92be0a0e7a55dfcd4d0374a6db93e624d1bc752dc8d1157906962ba874 WatchSource:0}: Error finding container 3ae18e92be0a0e7a55dfcd4d0374a6db93e624d1bc752dc8d1157906962ba874: Status 404 returned error can't find the container with id 3ae18e92be0a0e7a55dfcd4d0374a6db93e624d1bc752dc8d1157906962ba874
Jul 26 16:25:16.831670 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:16.831592    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" event={"ID":"c8ea1f90-cf56-416b-bd20-79301599a9ea","Type":"ContainerStarted","Data":"3ae18e92be0a0e7a55dfcd4d0374a6db93e624d1bc752dc8d1157906962ba874"}
Jul 26 16:25:19.444592 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:19.444546    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:25:19.444889 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:19.444625    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:25:19.444889 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:19.444654    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:25:19.847779 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:19.847746    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" event={"ID":"c8ea1f90-cf56-416b-bd20-79301599a9ea","Type":"ContainerStarted","Data":"11dcaf9f941772cd0fdeec1649ad028f6d6ee9c1160027a6103d6828b42dd9d4"}
Jul 26 16:25:19.868223 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:19.868164    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podStartSLOduration=1.101912643 podStartE2EDuration="3.868148516s" totalImagesPullingTime="2.766235873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:19.866275866 +0000 UTC m=+2819.050341999" watchObservedRunningTime="2026-07-26 16:25:19.868148516 +0000 UTC m=+2819.052214647"
Jul 26 16:25:20.539676 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:20.539643    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:20.544811 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:20.544788    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:20.852257 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:20.852161    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:20.853132 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:20.853111    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:22.932988 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:22.932950    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"]
Jul 26 16:25:22.937858 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:22.937841    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:22.940339 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:22.940315    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-8c3dba\""
Jul 26 16:25:22.948242 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:22.948204    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"]
Jul 26 16:25:23.060381 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.060348    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf5zp\" (UniqueName: \"kubernetes.io/projected/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-kube-api-access-gf5zp\") pod \"payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.060561 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.060421    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.161254 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.161191    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.161430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.161332    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf5zp\" (UniqueName: \"kubernetes.io/projected/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-kube-api-access-gf5zp\") pod \"payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.161909 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.161887    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.171153 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.171124    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf5zp\" (UniqueName: \"kubernetes.io/projected/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-kube-api-access-gf5zp\") pod \"payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.249713 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.249685    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:23.382494 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.382468    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"]
Jul 26 16:25:23.384125 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:25:23.384095    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd37c9f_218b_439d_b7a0_2ed0bec0b3b1.slice/crio-f87ec1c4527666aa7dc1ae45ee840d95e0e1205e4a9a91b636f1637e5f182029 WatchSource:0}: Error finding container f87ec1c4527666aa7dc1ae45ee840d95e0e1205e4a9a91b636f1637e5f182029: Status 404 returned error can't find the container with id f87ec1c4527666aa7dc1ae45ee840d95e0e1205e4a9a91b636f1637e5f182029
Jul 26 16:25:23.873764 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:23.873732    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg" event={"ID":"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1","Type":"ContainerStarted","Data":"f87ec1c4527666aa7dc1ae45ee840d95e0e1205e4a9a91b636f1637e5f182029"}
Jul 26 16:25:25.886949 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:25.886910    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg" event={"ID":"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1","Type":"ContainerStarted","Data":"3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9"}
Jul 26 16:25:25.887386 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:25.887011    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:25.904642 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:25.904592    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg" podStartSLOduration=2.334976975 podStartE2EDuration="3.904576547s" totalImagesPullingTime="1.569599572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:25:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:25:25.902603241 +0000 UTC m=+2825.086669388" watchObservedRunningTime="2026-07-26 16:25:25.904576547 +0000 UTC m=+2825.088642684"
Jul 26 16:25:28.390294 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:28.390258    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:25:35.939145 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:35.938016    2534 patch_prober.go:28] interesting pod/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": read tcp 10.134.0.2:59894->10.134.0.47:15021: read: connection reset by peer" start-of-body=
Jul 26 16:25:35.939145 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:35.938104    2534 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": read tcp 10.134.0.2:59894->10.134.0.47:15021: read: connection reset by peer"
Jul 26 16:25:36.894842 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.894804    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:25:36.952202 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.952170    2534 generic.go:356] "Generic (PLEG): container finished" podID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerID="11dcaf9f941772cd0fdeec1649ad028f6d6ee9c1160027a6103d6828b42dd9d4" exitCode=0
Jul 26 16:25:36.952687 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.952252    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" event={"ID":"c8ea1f90-cf56-416b-bd20-79301599a9ea","Type":"ContainerDied","Data":"11dcaf9f941772cd0fdeec1649ad028f6d6ee9c1160027a6103d6828b42dd9d4"}
Jul 26 16:25:36.952687 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.952672    2534 scope.go:117] "RemoveContainer" containerID="11dcaf9f941772cd0fdeec1649ad028f6d6ee9c1160027a6103d6828b42dd9d4"
Jul 26 16:25:36.953810 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.953782    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:25:36.953895 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.953855    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:25:36.953895 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:36.953890    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:25:37.960024 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:37.959989    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" event={"ID":"c8ea1f90-cf56-416b-bd20-79301599a9ea","Type":"ContainerStarted","Data":"1fc98c32c16098f7e0c0cbc2530d59c4f967fb2129c4e3e3668278962fb3adbe"}
Jul 26 16:25:38.540291 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:38.540223    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:38.540575 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:38.540551    2534 patch_prober.go:28] interesting pod/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 26 16:25:38.540634 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:38.540603    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 26 16:25:39.540048 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:39.540003    2534 patch_prober.go:28] interesting pod/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 26 16:25:39.540539 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:39.540080    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 26 16:25:40.539497 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:40.539461    2534 patch_prober.go:28] interesting pod/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 26 16:25:40.539673 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:40.539520    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 26 16:25:41.540117 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:41.540081    2534 patch_prober.go:28] interesting pod/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 26 16:25:41.540522 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:41.540133    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 26 16:25:42.543560 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:42.543530    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:42.543934 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:42.543857    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:25:42.544518 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:25:42.544502    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:26:02.180362 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.180327    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"]
Jul 26 16:26:02.184031 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.184007    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.186564 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.186545    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-10e721\"/\"openshift-service-ca.crt\""
Jul 26 16:26:02.187386 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.187371    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-10e721\"/\"kube-root-ca.crt\""
Jul 26 16:26:02.187456 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.187416    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-10e721\"/\"ipp-route-10e721-kserve-self-signed-certs\""
Jul 26 16:26:02.187573 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.187557    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-10e721\"/\"default-dockercfg-n77pg\""
Jul 26 16:26:02.194574 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.194547    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"]
Jul 26 16:26:02.348316 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.348284    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-home\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.348511 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.348321    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-dshm\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.348511 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.348362    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tmp-dir\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.348511 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.348466    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-268rg\" (UniqueName: \"kubernetes.io/projected/e29c5b81-22dc-43ce-a295-9588cf7c86a9-kube-api-access-268rg\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.348685 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.348515    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tls-certs\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.348685 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.348582    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-model-cache\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449032    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-268rg\" (UniqueName: \"kubernetes.io/projected/e29c5b81-22dc-43ce-a295-9588cf7c86a9-kube-api-access-268rg\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449081    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tls-certs\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449123    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-model-cache\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449170    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-home\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449195    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-dshm\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449260 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449244    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tmp-dir\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449587 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449561    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-model-cache\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449660 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449609    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-home\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.449720 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.449699    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tmp-dir\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.451815 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.451791    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tls-certs\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.451916 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.451831    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-dshm\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.457025 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.457008    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-268rg\" (UniqueName: \"kubernetes.io/projected/e29c5b81-22dc-43ce-a295-9588cf7c86a9-kube-api-access-268rg\") pod \"ipp-route-10e721-kserve-78889f79d7-26bbw\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") " pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.496087 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.496056    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:02.624174 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:02.624147    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"]
Jul 26 16:26:02.626125 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:26:02.626101    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode29c5b81_22dc_43ce_a295_9588cf7c86a9.slice/crio-642833a5d242accae0ee6841d101e28979dc76ce785e0df1dc40b57159c5590d WatchSource:0}: Error finding container 642833a5d242accae0ee6841d101e28979dc76ce785e0df1dc40b57159c5590d: Status 404 returned error can't find the container with id 642833a5d242accae0ee6841d101e28979dc76ce785e0df1dc40b57159c5590d
Jul 26 16:26:03.084787 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:03.084753    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw" event={"ID":"e29c5b81-22dc-43ce-a295-9588cf7c86a9","Type":"ContainerStarted","Data":"922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a"}
Jul 26 16:26:03.084787 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:03.084797    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw" event={"ID":"e29c5b81-22dc-43ce-a295-9588cf7c86a9","Type":"ContainerStarted","Data":"642833a5d242accae0ee6841d101e28979dc76ce785e0df1dc40b57159c5590d"}
Jul 26 16:26:03.101749 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:03.101698    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw" podStartSLOduration=1.101683 podStartE2EDuration="1.101683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:26:03.099696007 +0000 UTC m=+2862.283762144" watchObservedRunningTime="2026-07-26 16:26:03.101683 +0000 UTC m=+2862.285749136"
Jul 26 16:26:05.796279 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:05.796227    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:26:12.496540 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:12.496499    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:12.496962 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:12.496558    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:12.509198 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:12.509172    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:13.146095 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:13.146066    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:26:34.788033 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:34.787996    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:26:46.533989 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.533956    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"]
Jul 26 16:26:46.537686 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.537670    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.540151 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.540128    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-eb0742-openshift-default-dockercfg-b7ww6\""
Jul 26 16:26:46.549655 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.549632    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"]
Jul 26 16:26:46.588568 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588534    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588756 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588573    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltlkd\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-kube-api-access-ltlkd\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588756 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588601    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-envoy\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588756 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588644    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-token\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588756 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588708    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-socket\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588756 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588739    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-podinfo\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588756 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588758    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-certs\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588987 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588818    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-data\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.588987 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.588837    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-credential-socket\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.689739 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689701    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-data\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.689940 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689747    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-credential-socket\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.689940 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689801    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.689940 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689836    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltlkd\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-kube-api-access-ltlkd\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.689940 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689877    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-envoy\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.689940 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689901    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-token\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690216 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.689951    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-socket\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690216 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690006    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-podinfo\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690216 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690036    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-certs\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690354 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690208    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-data\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690354 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690269    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-credential-socket\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690541 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690522    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-certs\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690604 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690573    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-socket\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.690805 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.690781    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istiod-ca-cert\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.693077 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.693049    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-podinfo\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.693328 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.693174    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-envoy\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.697861 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.697832    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-token\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.698096 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.698079    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltlkd\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-kube-api-access-ltlkd\") pod \"e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") " pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.847867 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.847778    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:46.983577 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.983550    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"]
Jul 26 16:26:46.985617 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:26:46.985589    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ed2e71_e3b0_4544_a014_7ef3dffc2e5c.slice/crio-84e9782d93ad4ae0a1867fcc33faa71bb03b5443ad7104280b8f11505676528b WatchSource:0}: Error finding container 84e9782d93ad4ae0a1867fcc33faa71bb03b5443ad7104280b8f11505676528b: Status 404 returned error can't find the container with id 84e9782d93ad4ae0a1867fcc33faa71bb03b5443ad7104280b8f11505676528b
Jul 26 16:26:46.988305 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.988272    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:26:46.988382 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.988335    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:26:46.988382 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:46.988364    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:26:47.320272 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:47.320216    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" event={"ID":"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c","Type":"ContainerStarted","Data":"4c7ba265bc916f701ce2976eb810f3ebfcc8337dd539ace96f948587474abb73"}
Jul 26 16:26:47.320451 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:47.320297    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" event={"ID":"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c","Type":"ContainerStarted","Data":"84e9782d93ad4ae0a1867fcc33faa71bb03b5443ad7104280b8f11505676528b"}
Jul 26 16:26:47.340861 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:47.340773    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" podStartSLOduration=1.3407579 podStartE2EDuration="1.3407579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:26:47.339801023 +0000 UTC m=+2906.523867160" watchObservedRunningTime="2026-07-26 16:26:47.3407579 +0000 UTC m=+2906.524824037"
Jul 26 16:26:47.848304 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:47.848273    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:48.853243 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:48.853208    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:49.329802 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:49.329765    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:49.330778 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:49.330757    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:26:52.382340 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.382302    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"]
Jul 26 16:26:52.385954 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.385937    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.388428 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.388404    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-eb0742-dockercfg-2mqr7\""
Jul 26 16:26:52.388793 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.388773    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-eb0742\""
Jul 26 16:26:52.394402 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.394384    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"]
Jul 26 16:26:52.410917 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.410882    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"]
Jul 26 16:26:52.414457 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.414438    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.425106 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.425079    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"]
Jul 26 16:26:52.448545 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.448513    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6505111d-5fd6-4ba7-89a6-db37342fa89b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.448698 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.448618    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccs9v\" (UniqueName: \"kubernetes.io/projected/7ebff3ef-98b9-41af-9011-168ff5202695-kube-api-access-ccs9v\") pod \"payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.448698 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.448659    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ebff3ef-98b9-41af-9011-168ff5202695-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.448778 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.448737    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jnhf\" (UniqueName: \"kubernetes.io/projected/6505111d-5fd6-4ba7-89a6-db37342fa89b-kube-api-access-6jnhf\") pod \"payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.550093 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.550051    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ebff3ef-98b9-41af-9011-168ff5202695-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.550348 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.550115    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jnhf\" (UniqueName: \"kubernetes.io/projected/6505111d-5fd6-4ba7-89a6-db37342fa89b-kube-api-access-6jnhf\") pod \"payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.550348 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.550175    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6505111d-5fd6-4ba7-89a6-db37342fa89b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.550348 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.550216    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccs9v\" (UniqueName: \"kubernetes.io/projected/7ebff3ef-98b9-41af-9011-168ff5202695-kube-api-access-ccs9v\") pod \"payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.550945 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.550919    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ebff3ef-98b9-41af-9011-168ff5202695-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.551036 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.550958    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6505111d-5fd6-4ba7-89a6-db37342fa89b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.558206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.558178    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jnhf\" (UniqueName: \"kubernetes.io/projected/6505111d-5fd6-4ba7-89a6-db37342fa89b-kube-api-access-6jnhf\") pod \"payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.558363 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.558327    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccs9v\" (UniqueName: \"kubernetes.io/projected/7ebff3ef-98b9-41af-9011-168ff5202695-kube-api-access-ccs9v\") pod \"payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.698423 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.698381    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:52.725019 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.724985    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:52.872192 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:52.872164    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"]
Jul 26 16:26:52.874268 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:26:52.874224    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebff3ef_98b9_41af_9011_168ff5202695.slice/crio-aa4e51de6bca1498f0ebf7496f6afed18a31c6ed50bafaf47a6e8c259f365f7a WatchSource:0}: Error finding container aa4e51de6bca1498f0ebf7496f6afed18a31c6ed50bafaf47a6e8c259f365f7a: Status 404 returned error can't find the container with id aa4e51de6bca1498f0ebf7496f6afed18a31c6ed50bafaf47a6e8c259f365f7a
Jul 26 16:26:53.051506 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.051479    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"]
Jul 26 16:26:53.053311 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:26:53.053284    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6505111d_5fd6_4ba7_89a6_db37342fa89b.slice/crio-ac412637e4acd79da1face78ab60ffd34a6d909e7c823aad3240a0df026cea13 WatchSource:0}: Error finding container ac412637e4acd79da1face78ab60ffd34a6d909e7c823aad3240a0df026cea13: Status 404 returned error can't find the container with id ac412637e4acd79da1face78ab60ffd34a6d909e7c823aad3240a0df026cea13
Jul 26 16:26:53.351212 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.351112    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq" event={"ID":"6505111d-5fd6-4ba7-89a6-db37342fa89b","Type":"ContainerStarted","Data":"4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498"}
Jul 26 16:26:53.351212 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.351172    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq" event={"ID":"6505111d-5fd6-4ba7-89a6-db37342fa89b","Type":"ContainerStarted","Data":"ac412637e4acd79da1face78ab60ffd34a6d909e7c823aad3240a0df026cea13"}
Jul 26 16:26:53.351212 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.351202    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:26:53.352623 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.352593    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6" event={"ID":"7ebff3ef-98b9-41af-9011-168ff5202695","Type":"ContainerStarted","Data":"429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf"}
Jul 26 16:26:53.352732 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.352632    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6" event={"ID":"7ebff3ef-98b9-41af-9011-168ff5202695","Type":"ContainerStarted","Data":"aa4e51de6bca1498f0ebf7496f6afed18a31c6ed50bafaf47a6e8c259f365f7a"}
Jul 26 16:26:53.352732 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.352714    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:26:53.366779 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.366731    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq" podStartSLOduration=1.36671211 podStartE2EDuration="1.36671211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:26:53.365021368 +0000 UTC m=+2912.549087504" watchObservedRunningTime="2026-07-26 16:26:53.36671211 +0000 UTC m=+2912.550778248"
Jul 26 16:26:53.380823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:26:53.380776    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6" podStartSLOduration=1.380765393 podStartE2EDuration="1.380765393s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:26:53.37852376 +0000 UTC m=+2912.562589898" watchObservedRunningTime="2026-07-26 16:26:53.380765393 +0000 UTC m=+2912.564831529"
Jul 26 16:27:00.089695 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:00.089663    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:27:04.360210 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:04.360180    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:27:04.360677 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:04.360358    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:27:08.452475 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:08.452441    2534 generic.go:356] "Generic (PLEG): container finished" podID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerID="4c7ba265bc916f701ce2976eb810f3ebfcc8337dd539ace96f948587474abb73" exitCode=0
Jul 26 16:27:08.453157 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:08.452517    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" event={"ID":"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c","Type":"ContainerDied","Data":"4c7ba265bc916f701ce2976eb810f3ebfcc8337dd539ace96f948587474abb73"}
Jul 26 16:27:08.453157 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:08.452898    2534 scope.go:117] "RemoveContainer" containerID="4c7ba265bc916f701ce2976eb810f3ebfcc8337dd539ace96f948587474abb73"
Jul 26 16:27:08.453942 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:08.453908    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:27:08.454034 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:08.453977    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:27:08.454034 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:08.454011    2534 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 26 16:27:09.459758 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:09.459723    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" event={"ID":"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c","Type":"ContainerStarted","Data":"6715a8a816d633306b12d7355447f6948207a66270ed3d2603d29ebb3e640966"}
Jul 26 16:27:09.848833 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:09.848733    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:27:09.848987 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:09.848927    2534 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 26 16:27:09.849031 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:09.848976    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 26 16:27:10.848888 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:10.848855    2534 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 26 16:27:10.849301 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:10.848911    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 26 16:27:11.848798 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:11.848762    2534 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 26 16:27:11.848970 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:11.848817    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 26 16:27:12.849129 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:12.849090    2534 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 26 16:27:12.849550 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:12.849149    2534 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 26 16:27:13.851448 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:13.851411    2534 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:27:13.851863 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:13.851655    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:27:13.852255 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:13.852221    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:27:21.432705 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.432671    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"]
Jul 26 16:27:21.433167 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.432922    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6" podUID="7ebff3ef-98b9-41af-9011-168ff5202695" containerName="payload-processing" containerID="cri-o://429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf" gracePeriod=30
Jul 26 16:27:21.442427 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.442399    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"]
Jul 26 16:27:21.442666 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.442619    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq" podUID="6505111d-5fd6-4ba7-89a6-db37342fa89b" containerName="payload-pre-processing" containerID="cri-o://4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498" gracePeriod=30
Jul 26 16:27:21.691888 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.691834    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:27:21.722725 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.722700    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:27:21.835764 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.835731    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccs9v\" (UniqueName: \"kubernetes.io/projected/7ebff3ef-98b9-41af-9011-168ff5202695-kube-api-access-ccs9v\") pod \"7ebff3ef-98b9-41af-9011-168ff5202695\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") "
Jul 26 16:27:21.835764 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.835771    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6505111d-5fd6-4ba7-89a6-db37342fa89b-plugins-config-volume\") pod \"6505111d-5fd6-4ba7-89a6-db37342fa89b\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") "
Jul 26 16:27:21.836025 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.835807    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ebff3ef-98b9-41af-9011-168ff5202695-plugins-config-volume\") pod \"7ebff3ef-98b9-41af-9011-168ff5202695\" (UID: \"7ebff3ef-98b9-41af-9011-168ff5202695\") "
Jul 26 16:27:21.836025 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.835897    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jnhf\" (UniqueName: \"kubernetes.io/projected/6505111d-5fd6-4ba7-89a6-db37342fa89b-kube-api-access-6jnhf\") pod \"6505111d-5fd6-4ba7-89a6-db37342fa89b\" (UID: \"6505111d-5fd6-4ba7-89a6-db37342fa89b\") "
Jul 26 16:27:21.836226 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.836200    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6505111d-5fd6-4ba7-89a6-db37342fa89b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6505111d-5fd6-4ba7-89a6-db37342fa89b" (UID: "6505111d-5fd6-4ba7-89a6-db37342fa89b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:21.836311 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.836200    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ebff3ef-98b9-41af-9011-168ff5202695-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7ebff3ef-98b9-41af-9011-168ff5202695" (UID: "7ebff3ef-98b9-41af-9011-168ff5202695"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:21.838003 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.837979    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6505111d-5fd6-4ba7-89a6-db37342fa89b-kube-api-access-6jnhf" (OuterVolumeSpecName: "kube-api-access-6jnhf") pod "6505111d-5fd6-4ba7-89a6-db37342fa89b" (UID: "6505111d-5fd6-4ba7-89a6-db37342fa89b"). InnerVolumeSpecName "kube-api-access-6jnhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:21.838074 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.838024    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ebff3ef-98b9-41af-9011-168ff5202695-kube-api-access-ccs9v" (OuterVolumeSpecName: "kube-api-access-ccs9v") pod "7ebff3ef-98b9-41af-9011-168ff5202695" (UID: "7ebff3ef-98b9-41af-9011-168ff5202695"). InnerVolumeSpecName "kube-api-access-ccs9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:21.937307 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.937271    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccs9v\" (UniqueName: \"kubernetes.io/projected/7ebff3ef-98b9-41af-9011-168ff5202695-kube-api-access-ccs9v\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:21.937307 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.937301    2534 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6505111d-5fd6-4ba7-89a6-db37342fa89b-plugins-config-volume\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:21.937307 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.937313    2534 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ebff3ef-98b9-41af-9011-168ff5202695-plugins-config-volume\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:21.937532 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:21.937324    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jnhf\" (UniqueName: \"kubernetes.io/projected/6505111d-5fd6-4ba7-89a6-db37342fa89b-kube-api-access-6jnhf\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:22.526180 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.526146    2534 generic.go:356] "Generic (PLEG): container finished" podID="7ebff3ef-98b9-41af-9011-168ff5202695" containerID="429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf" exitCode=0
Jul 26 16:27:22.526639 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.526187    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"
Jul 26 16:27:22.526639 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.526225    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6" event={"ID":"7ebff3ef-98b9-41af-9011-168ff5202695","Type":"ContainerDied","Data":"429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf"}
Jul 26 16:27:22.526639 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.526298    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6" event={"ID":"7ebff3ef-98b9-41af-9011-168ff5202695","Type":"ContainerDied","Data":"aa4e51de6bca1498f0ebf7496f6afed18a31c6ed50bafaf47a6e8c259f365f7a"}
Jul 26 16:27:22.526639 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.526324    2534 scope.go:117] "RemoveContainer" containerID="429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf"
Jul 26 16:27:22.527749 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.527720    2534 generic.go:356] "Generic (PLEG): container finished" podID="6505111d-5fd6-4ba7-89a6-db37342fa89b" containerID="4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498" exitCode=0
Jul 26 16:27:22.527867 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.527761    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"
Jul 26 16:27:22.527867 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.527815    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq" event={"ID":"6505111d-5fd6-4ba7-89a6-db37342fa89b","Type":"ContainerDied","Data":"4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498"}
Jul 26 16:27:22.527973 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.527872    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq" event={"ID":"6505111d-5fd6-4ba7-89a6-db37342fa89b","Type":"ContainerDied","Data":"ac412637e4acd79da1face78ab60ffd34a6d909e7c823aad3240a0df026cea13"}
Jul 26 16:27:22.536973 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.536959    2534 scope.go:117] "RemoveContainer" containerID="429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf"
Jul 26 16:27:22.537261 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:27:22.537220    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf\": container with ID starting with 429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf not found: ID does not exist" containerID="429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf"
Jul 26 16:27:22.537414 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.537390    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf"} err="failed to get container status \"429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf\": rpc error: code = NotFound desc = could not find container \"429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf\": container with ID starting with 429823090e1f9aaeff0047187d886d3bdc3ab52f274e71145f7461f4df221cdf not found: ID does not exist"
Jul 26 16:27:22.537514 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.537417    2534 scope.go:117] "RemoveContainer" containerID="4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498"
Jul 26 16:27:22.544928 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.544912    2534 scope.go:117] "RemoveContainer" containerID="4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498"
Jul 26 16:27:22.545172 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:27:22.545150    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498\": container with ID starting with 4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498 not found: ID does not exist" containerID="4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498"
Jul 26 16:27:22.545257 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.545182    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498"} err="failed to get container status \"4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498\": rpc error: code = NotFound desc = could not find container \"4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498\": container with ID starting with 4f7f00fd8c431b9f9f76587ce3308d7ebd4b72f71dde00ffea3e36782b32f498 not found: ID does not exist"
Jul 26 16:27:22.547529 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.547508    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"]
Jul 26 16:27:22.553533 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.553512    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eb0742-687657c7fb-tktg6"]
Jul 26 16:27:22.562394 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.562374    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"]
Jul 26 16:27:22.568036 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:22.568002    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-eb0742-7c75c67776-4lssq"]
Jul 26 16:27:23.294553 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:23.294519    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6505111d-5fd6-4ba7-89a6-db37342fa89b" path="/var/lib/kubelet/pods/6505111d-5fd6-4ba7-89a6-db37342fa89b/volumes"
Jul 26 16:27:23.294917 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:23.294904    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ebff3ef-98b9-41af-9011-168ff5202695" path="/var/lib/kubelet/pods/7ebff3ef-98b9-41af-9011-168ff5202695/volumes"
Jul 26 16:27:25.887875 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:25.887841    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:27:31.456477 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:31.456444    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"]
Jul 26 16:27:31.456898 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:31.456672    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy" containerID="cri-o://6715a8a816d633306b12d7355447f6948207a66270ed3d2603d29ebb3e640966" gracePeriod=30
Jul 26 16:27:35.038511 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038478    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"]
Jul 26 16:27:35.038886 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038874    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6505111d-5fd6-4ba7-89a6-db37342fa89b" containerName="payload-pre-processing"
Jul 26 16:27:35.038886 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038885    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6505111d-5fd6-4ba7-89a6-db37342fa89b" containerName="payload-pre-processing"
Jul 26 16:27:35.038957 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038897    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebff3ef-98b9-41af-9011-168ff5202695" containerName="payload-processing"
Jul 26 16:27:35.038957 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038903    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebff3ef-98b9-41af-9011-168ff5202695" containerName="payload-processing"
Jul 26 16:27:35.039020 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038967    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="6505111d-5fd6-4ba7-89a6-db37342fa89b" containerName="payload-pre-processing"
Jul 26 16:27:35.039020 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.038977    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ebff3ef-98b9-41af-9011-168ff5202695" containerName="payload-processing"
Jul 26 16:27:35.042004 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.041986    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:35.044573 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.044558    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:27:35.044573 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.044564    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:27:35.053464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.053443    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"]
Jul 26 16:27:35.073971 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.073945    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96efc24b-4b0b-4660-9ba3-a87331a4835d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk\" (UID: \"96efc24b-4b0b-4660-9ba3-a87331a4835d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:35.175058 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.175035    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96efc24b-4b0b-4660-9ba3-a87331a4835d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk\" (UID: \"96efc24b-4b0b-4660-9ba3-a87331a4835d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:35.175657 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.175638    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96efc24b-4b0b-4660-9ba3-a87331a4835d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk\" (UID: \"96efc24b-4b0b-4660-9ba3-a87331a4835d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:35.352762 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.352690    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:35.486980 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.486933    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"]
Jul 26 16:27:35.489718 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:27:35.489683    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96efc24b_4b0b_4660_9ba3_a87331a4835d.slice/crio-eb85fa0e51fd5a925c357004091151fd14a144220e1310eb36e8d5d55b858a3a WatchSource:0}: Error finding container eb85fa0e51fd5a925c357004091151fd14a144220e1310eb36e8d5d55b858a3a: Status 404 returned error can't find the container with id eb85fa0e51fd5a925c357004091151fd14a144220e1310eb36e8d5d55b858a3a
Jul 26 16:27:35.596913 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:35.596883    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk" event={"ID":"96efc24b-4b0b-4660-9ba3-a87331a4835d","Type":"ContainerStarted","Data":"eb85fa0e51fd5a925c357004091151fd14a144220e1310eb36e8d5d55b858a3a"}
Jul 26 16:27:36.603811 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.603776    2534 generic.go:356] "Generic (PLEG): container finished" podID="96efc24b-4b0b-4660-9ba3-a87331a4835d" containerID="9b96e38ba1a6029dad04150ad768ea749c8ede9042c40133713e9c2b7d5cbfb7" exitCode=0
Jul 26 16:27:36.604274 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.603826    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk" event={"ID":"96efc24b-4b0b-4660-9ba3-a87331a4835d","Type":"ContainerDied","Data":"9b96e38ba1a6029dad04150ad768ea749c8ede9042c40133713e9c2b7d5cbfb7"}
Jul 26 16:27:36.606206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.606177    2534 generic.go:356] "Generic (PLEG): container finished" podID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerID="6715a8a816d633306b12d7355447f6948207a66270ed3d2603d29ebb3e640966" exitCode=0
Jul 26 16:27:36.606335 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.606279    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" event={"ID":"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c","Type":"ContainerDied","Data":"6715a8a816d633306b12d7355447f6948207a66270ed3d2603d29ebb3e640966"}
Jul 26 16:27:36.606380 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.606361    2534 scope.go:117] "RemoveContainer" containerID="4c7ba265bc916f701ce2976eb810f3ebfcc8337dd539ace96f948587474abb73"
Jul 26 16:27:36.718916 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.718863    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:27:36.790662 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790624    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-certs\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.790662 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790667    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-socket\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.790893 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790690    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-data\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.790893 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790717    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istiod-ca-cert\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.790893 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790811    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-podinfo\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.790893 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790856    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-token\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.791107 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790915    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:27:36.791107 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.790990    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:27:36.791221 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791184    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:36.791344 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791326    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-credential-socket\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.791409 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791389    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltlkd\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-kube-api-access-ltlkd\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.791461 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791427    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-envoy\") pod \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\" (UID: \"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c\") "
Jul 26 16:27:36.791567 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791545    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-data" (OuterVolumeSpecName: "istio-data") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:27:36.791638 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791609    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:27:36.791789 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791768    2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-credential-socket\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.791789 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791790    2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.791944 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791803    2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-workload-socket\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.791944 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791815    2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-data\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.791944 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.791826    2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istiod-ca-cert\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.793253 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.793215    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 16:27:36.793415 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.793395    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-kube-api-access-ltlkd" (OuterVolumeSpecName: "kube-api-access-ltlkd") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "kube-api-access-ltlkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:36.793511 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.793491    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-token" (OuterVolumeSpecName: "istio-token") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:27:36.793702 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.793680    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" (UID: "77ed2e71-e3b0-4544-a014-7ef3dffc2e5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:27:36.892597 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.892575    2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-podinfo\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.892700 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.892599    2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-token\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.892700 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.892614    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltlkd\" (UniqueName: \"kubernetes.io/projected/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-kube-api-access-ltlkd\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:36.892700 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:36.892627    2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c-istio-envoy\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:37.615034 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.614940    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6" event={"ID":"77ed2e71-e3b0-4544-a014-7ef3dffc2e5c","Type":"ContainerDied","Data":"84e9782d93ad4ae0a1867fcc33faa71bb03b5443ad7104280b8f11505676528b"}
Jul 26 16:27:37.615034 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.614970    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"
Jul 26 16:27:37.615034 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.615028    2534 scope.go:117] "RemoveContainer" containerID="6715a8a816d633306b12d7355447f6948207a66270ed3d2603d29ebb3e640966"
Jul 26 16:27:37.633166 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.633138    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"]
Jul 26 16:27:37.639337 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.639304    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-eb0742-openshift-default-5f8f5546b4-w77b6"]
Jul 26 16:27:37.754669 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.754646    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:37.802493 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.802465    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96efc24b-4b0b-4660-9ba3-a87331a4835d-maas-api-service-ca\") pod \"96efc24b-4b0b-4660-9ba3-a87331a4835d\" (UID: \"96efc24b-4b0b-4660-9ba3-a87331a4835d\") "
Jul 26 16:27:37.802801 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.802779    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96efc24b-4b0b-4660-9ba3-a87331a4835d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96efc24b-4b0b-4660-9ba3-a87331a4835d" (UID: "96efc24b-4b0b-4660-9ba3-a87331a4835d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:27:37.903615 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:37.903548    2534 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96efc24b-4b0b-4660-9ba3-a87331a4835d-maas-api-service-ca\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:27:38.622588 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:38.622557    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"
Jul 26 16:27:38.623065 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:38.622596    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk" event={"ID":"96efc24b-4b0b-4660-9ba3-a87331a4835d","Type":"ContainerDied","Data":"eb85fa0e51fd5a925c357004091151fd14a144220e1310eb36e8d5d55b858a3a"}
Jul 26 16:27:38.623065 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:38.622653    2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb85fa0e51fd5a925c357004091151fd14a144220e1310eb36e8d5d55b858a3a"
Jul 26 16:27:39.294004 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:39.293972    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" path="/var/lib/kubelet/pods/77ed2e71-e3b0-4544-a014-7ef3dffc2e5c/volumes"
Jul 26 16:27:49.888757 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:49.888674    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:27:55.122529 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:55.122491    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"]
Jul 26 16:27:55.128409 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:55.128383    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-10e721-m4dxk"]
Jul 26 16:27:55.294426 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:27:55.294397    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96efc24b-4b0b-4660-9ba3-a87331a4835d" path="/var/lib/kubelet/pods/96efc24b-4b0b-4660-9ba3-a87331a4835d/volumes"
Jul 26 16:28:00.476605 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:00.476570    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"]
Jul 26 16:28:03.907459 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:03.907420    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:28:15.495532 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.495494    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw" podUID="e29c5b81-22dc-43ce-a295-9588cf7c86a9" containerName="main" containerID="cri-o://922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a" gracePeriod=45
Jul 26 16:28:15.753958 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.753895    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:28:15.822908 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.822877    2534 generic.go:356] "Generic (PLEG): container finished" podID="e29c5b81-22dc-43ce-a295-9588cf7c86a9" containerID="922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a" exitCode=0
Jul 26 16:28:15.823059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.822923    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"
Jul 26 16:28:15.823059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.822932    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw" event={"ID":"e29c5b81-22dc-43ce-a295-9588cf7c86a9","Type":"ContainerDied","Data":"922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a"}
Jul 26 16:28:15.823059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.822965    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw" event={"ID":"e29c5b81-22dc-43ce-a295-9588cf7c86a9","Type":"ContainerDied","Data":"642833a5d242accae0ee6841d101e28979dc76ce785e0df1dc40b57159c5590d"}
Jul 26 16:28:15.823059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.822981    2534 scope.go:117] "RemoveContainer" containerID="922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a"
Jul 26 16:28:15.831706 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.831686    2534 scope.go:117] "RemoveContainer" containerID="922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a"
Jul 26 16:28:15.831954 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:28:15.831935    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a\": container with ID starting with 922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a not found: ID does not exist" containerID="922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a"
Jul 26 16:28:15.832010 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.831963    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a"} err="failed to get container status \"922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a\": rpc error: code = NotFound desc = could not find container \"922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a\": container with ID starting with 922de9cbc8458678262e330dfd3985fc97709fc4c4fb3ff9cb451824791b280a not found: ID does not exist"
Jul 26 16:28:15.872791 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.872764    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-home\") pod \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") "
Jul 26 16:28:15.872903 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.872852    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tls-certs\") pod \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") "
Jul 26 16:28:15.872903 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.872882    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-dshm\") pod \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") "
Jul 26 16:28:15.872999 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.872963    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-model-cache\") pod \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") "
Jul 26 16:28:15.873055 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873009    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tmp-dir\") pod \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") "
Jul 26 16:28:15.873055 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873025    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-home" (OuterVolumeSpecName: "home") pod "e29c5b81-22dc-43ce-a295-9588cf7c86a9" (UID: "e29c5b81-22dc-43ce-a295-9588cf7c86a9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:28:15.873055 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873039    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-268rg\" (UniqueName: \"kubernetes.io/projected/e29c5b81-22dc-43ce-a295-9588cf7c86a9-kube-api-access-268rg\") pod \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\" (UID: \"e29c5b81-22dc-43ce-a295-9588cf7c86a9\") "
Jul 26 16:28:15.873280 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873258    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-model-cache" (OuterVolumeSpecName: "model-cache") pod "e29c5b81-22dc-43ce-a295-9588cf7c86a9" (UID: "e29c5b81-22dc-43ce-a295-9588cf7c86a9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:28:15.873365 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873278    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e29c5b81-22dc-43ce-a295-9588cf7c86a9" (UID: "e29c5b81-22dc-43ce-a295-9588cf7c86a9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:28:15.873417 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873380    2534 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-model-cache\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:15.873417 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873399    2534 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tmp-dir\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:15.873417 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.873413    2534 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-home\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:15.875146 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.875109    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-dshm" (OuterVolumeSpecName: "dshm") pod "e29c5b81-22dc-43ce-a295-9588cf7c86a9" (UID: "e29c5b81-22dc-43ce-a295-9588cf7c86a9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:28:15.875284 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.875221    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29c5b81-22dc-43ce-a295-9588cf7c86a9-kube-api-access-268rg" (OuterVolumeSpecName: "kube-api-access-268rg") pod "e29c5b81-22dc-43ce-a295-9588cf7c86a9" (UID: "e29c5b81-22dc-43ce-a295-9588cf7c86a9"). InnerVolumeSpecName "kube-api-access-268rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:28:15.875351 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.875295    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e29c5b81-22dc-43ce-a295-9588cf7c86a9" (UID: "e29c5b81-22dc-43ce-a295-9588cf7c86a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 16:28:15.974048 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.974022    2534 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e29c5b81-22dc-43ce-a295-9588cf7c86a9-tls-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:15.974048 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.974044    2534 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e29c5b81-22dc-43ce-a295-9588cf7c86a9-dshm\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:15.974188 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:15.974053    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-268rg\" (UniqueName: \"kubernetes.io/projected/e29c5b81-22dc-43ce-a295-9588cf7c86a9-kube-api-access-268rg\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:16.144059 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:16.144031    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"]
Jul 26 16:28:16.148038 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:16.148015    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-10e721/ipp-route-10e721-kserve-78889f79d7-26bbw"]
Jul 26 16:28:17.295038 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:17.295000    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e29c5b81-22dc-43ce-a295-9588cf7c86a9" path="/var/lib/kubelet/pods/e29c5b81-22dc-43ce-a295-9588cf7c86a9/volumes"
Jul 26 16:28:43.991151 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991121    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"]
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991508    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy"
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991519    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy"
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991532    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy"
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991537    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy"
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991553    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e29c5b81-22dc-43ce-a295-9588cf7c86a9" containerName="main"
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991558    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="e29c5b81-22dc-43ce-a295-9588cf7c86a9" containerName="main"
Jul 26 16:28:43.991571 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991575    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96efc24b-4b0b-4660-9ba3-a87331a4835d" containerName="revoke-keys"
Jul 26 16:28:43.991823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991580    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="96efc24b-4b0b-4660-9ba3-a87331a4835d" containerName="revoke-keys"
Jul 26 16:28:43.991823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991649    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="e29c5b81-22dc-43ce-a295-9588cf7c86a9" containerName="main"
Jul 26 16:28:43.991823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991657    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy"
Jul 26 16:28:43.991823 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.991662    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="96efc24b-4b0b-4660-9ba3-a87331a4835d" containerName="revoke-keys"
Jul 26 16:28:43.994867 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.994850    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:43.997386 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.997370    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:28:43.997450 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:43.997426    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:28:44.003582 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.003554    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"]
Jul 26 16:28:44.038035 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.038006    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861524e-a8fb-4bb9-afd6-98db8828f099-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw\" (UID: \"a861524e-a8fb-4bb9-afd6-98db8828f099\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:44.138575 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.138538    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861524e-a8fb-4bb9-afd6-98db8828f099-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw\" (UID: \"a861524e-a8fb-4bb9-afd6-98db8828f099\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:44.139132 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.139112    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861524e-a8fb-4bb9-afd6-98db8828f099-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw\" (UID: \"a861524e-a8fb-4bb9-afd6-98db8828f099\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:44.306500 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.306411    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:44.442960 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.442934    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"]
Jul 26 16:28:44.444869 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:28:44.444843    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda861524e_a8fb_4bb9_afd6_98db8828f099.slice/crio-49e16882f5b58e786127a18f77fc50424034af60c6074d685a5fd487c9e31442 WatchSource:0}: Error finding container 49e16882f5b58e786127a18f77fc50424034af60c6074d685a5fd487c9e31442: Status 404 returned error can't find the container with id 49e16882f5b58e786127a18f77fc50424034af60c6074d685a5fd487c9e31442
Jul 26 16:28:44.968360 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.968321    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw" event={"ID":"a861524e-a8fb-4bb9-afd6-98db8828f099","Type":"ContainerStarted","Data":"d60944b8f2a0e5a6070c558df5501dcae4055c1bd6f37e9ad93dcb83ce53e602"}
Jul 26 16:28:44.968531 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.968380    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw" event={"ID":"a861524e-a8fb-4bb9-afd6-98db8828f099","Type":"ContainerStarted","Data":"49e16882f5b58e786127a18f77fc50424034af60c6074d685a5fd487c9e31442"}
Jul 26 16:28:44.983407 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:44.983358    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw" podStartSLOduration=1.983342197 podStartE2EDuration="1.983342197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 16:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:28:44.981478429 +0000 UTC m=+3024.165544566" watchObservedRunningTime="2026-07-26 16:28:44.983342197 +0000 UTC m=+3024.167408334"
Jul 26 16:28:45.973916 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:45.973883    2534 generic.go:356] "Generic (PLEG): container finished" podID="a861524e-a8fb-4bb9-afd6-98db8828f099" containerID="d60944b8f2a0e5a6070c558df5501dcae4055c1bd6f37e9ad93dcb83ce53e602" exitCode=0
Jul 26 16:28:45.974322 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:45.973964    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw" event={"ID":"a861524e-a8fb-4bb9-afd6-98db8828f099","Type":"ContainerDied","Data":"d60944b8f2a0e5a6070c558df5501dcae4055c1bd6f37e9ad93dcb83ce53e602"}
Jul 26 16:28:47.114935 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.114907    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:47.170212 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.170178    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861524e-a8fb-4bb9-afd6-98db8828f099-maas-api-service-ca\") pod \"a861524e-a8fb-4bb9-afd6-98db8828f099\" (UID: \"a861524e-a8fb-4bb9-afd6-98db8828f099\") "
Jul 26 16:28:47.170554 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.170531    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a861524e-a8fb-4bb9-afd6-98db8828f099-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a861524e-a8fb-4bb9-afd6-98db8828f099" (UID: "a861524e-a8fb-4bb9-afd6-98db8828f099"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:28:47.271856 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.271774    2534 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a861524e-a8fb-4bb9-afd6-98db8828f099-maas-api-service-ca\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:47.985479 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.985444    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw" event={"ID":"a861524e-a8fb-4bb9-afd6-98db8828f099","Type":"ContainerDied","Data":"49e16882f5b58e786127a18f77fc50424034af60c6074d685a5fd487c9e31442"}
Jul 26 16:28:47.985647 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.985492    2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49e16882f5b58e786127a18f77fc50424034af60c6074d685a5fd487c9e31442"
Jul 26 16:28:47.985647 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:47.985457    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"
Jul 26 16:28:55.654119 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:55.654084    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"]
Jul 26 16:28:55.654632 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:55.654358    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg" podUID="6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" containerName="payload-pre-processing" containerID="cri-o://3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9" gracePeriod=30
Jul 26 16:28:55.908613 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:55.908555    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:28:56.025877 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.025843    2534 generic.go:356] "Generic (PLEG): container finished" podID="6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" containerID="3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9" exitCode=0
Jul 26 16:28:56.026056 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.025886    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"
Jul 26 16:28:56.026056 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.025910    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg" event={"ID":"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1","Type":"ContainerDied","Data":"3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9"}
Jul 26 16:28:56.026056 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.025958    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg" event={"ID":"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1","Type":"ContainerDied","Data":"f87ec1c4527666aa7dc1ae45ee840d95e0e1205e4a9a91b636f1637e5f182029"}
Jul 26 16:28:56.026056 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.025981    2534 scope.go:117] "RemoveContainer" containerID="3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9"
Jul 26 16:28:56.034275 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.034251    2534 scope.go:117] "RemoveContainer" containerID="3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9"
Jul 26 16:28:56.034552 ip-10-0-138-109 kubenswrapper[2534]: E0726 16:28:56.034536    2534 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9\": container with ID starting with 3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9 not found: ID does not exist" containerID="3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9"
Jul 26 16:28:56.034606 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.034561    2534 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9"} err="failed to get container status \"3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9\": rpc error: code = NotFound desc = could not find container \"3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9\": container with ID starting with 3839c13de015d88b405d69336f6169897351954f0fcf5c3b8a0d97e9132b76b9 not found: ID does not exist"
Jul 26 16:28:56.052054 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.052023    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-plugins-config-volume\") pod \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") "
Jul 26 16:28:56.052143 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.052132    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf5zp\" (UniqueName: \"kubernetes.io/projected/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-kube-api-access-gf5zp\") pod \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\" (UID: \"6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1\") "
Jul 26 16:28:56.052366 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.052348    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" (UID: "6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:28:56.054062 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.054043    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-kube-api-access-gf5zp" (OuterVolumeSpecName: "kube-api-access-gf5zp") pod "6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" (UID: "6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1"). InnerVolumeSpecName "kube-api-access-gf5zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:28:56.153447 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.153400    2534 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-plugins-config-volume\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.153447 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.153442    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf5zp\" (UniqueName: \"kubernetes.io/projected/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1-kube-api-access-gf5zp\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:28:56.348005 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.347970    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"]
Jul 26 16:28:56.351860 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:56.351831    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-8c3dba-6bd5d6d7dd-xl7kg"]
Jul 26 16:28:57.294409 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:57.294374    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" path="/var/lib/kubelet/pods/6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1/volumes"
Jul 26 16:28:58.590645 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:58.590611    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:28:59.080857 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:59.080822    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"]
Jul 26 16:28:59.087522 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:59.087498    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-8c3dba-g9hrw"]
Jul 26 16:28:59.294114 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:28:59.294082    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a861524e-a8fb-4bb9-afd6-98db8828f099" path="/var/lib/kubelet/pods/a861524e-a8fb-4bb9-afd6-98db8828f099/volumes"
Jul 26 16:29:05.894333 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:05.894292    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"]
Jul 26 16:29:05.894831 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:05.894578    2534 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy" containerID="cri-o://1fc98c32c16098f7e0c0cbc2530d59c4f967fb2129c4e3e3668278962fb3adbe" gracePeriod=30
Jul 26 16:29:11.101572 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.101529    2534 generic.go:356] "Generic (PLEG): container finished" podID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerID="1fc98c32c16098f7e0c0cbc2530d59c4f967fb2129c4e3e3668278962fb3adbe" exitCode=0
Jul 26 16:29:11.102071 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.101600    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" event={"ID":"c8ea1f90-cf56-416b-bd20-79301599a9ea","Type":"ContainerDied","Data":"1fc98c32c16098f7e0c0cbc2530d59c4f967fb2129c4e3e3668278962fb3adbe"}
Jul 26 16:29:11.102071 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.101671    2534 scope.go:117] "RemoveContainer" containerID="11dcaf9f941772cd0fdeec1649ad028f6d6ee9c1160027a6103d6828b42dd9d4"
Jul 26 16:29:11.153987 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.153946    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:29:11.314961 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.314930    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-credential-socket\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.314961 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.314967    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-data\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315000    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-certs\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315020    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea1f90-cf56-416b-bd20-79301599a9ea-istiod-ca-cert\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315045    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9kvq\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-kube-api-access-f9kvq\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315084    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-socket\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315167    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-podinfo\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315206 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315200    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-token\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315227    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-envoy\") pod \"c8ea1f90-cf56-416b-bd20-79301599a9ea\" (UID: \"c8ea1f90-cf56-416b-bd20-79301599a9ea\") "
Jul 26 16:29:11.315544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315271    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:29:11.315544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315371    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:29:11.315544 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315506    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8ea1f90-cf56-416b-bd20-79301599a9ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:11.315735 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315588    2534 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-certs\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.315735 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315606    2534 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8ea1f90-cf56-416b-bd20-79301599a9ea-istiod-ca-cert\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.315735 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315619    2534 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-credential-socket\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.315886 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315739    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:29:11.315886 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.315807    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:29:11.317492 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.317462    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 16:29:11.317602 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.317517    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 26 16:29:11.317757 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.317743    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:11.317819 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.317791    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-kube-api-access-f9kvq" (OuterVolumeSpecName: "kube-api-access-f9kvq") pod "c8ea1f90-cf56-416b-bd20-79301599a9ea" (UID: "c8ea1f90-cf56-416b-bd20-79301599a9ea"). InnerVolumeSpecName "kube-api-access-f9kvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 16:29:11.416430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.416375    2534 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-data\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.416430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.416403    2534 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9kvq\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-kube-api-access-f9kvq\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.416430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.416415    2534 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-workload-socket\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.416430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.416425    2534 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-podinfo\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.416430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.416434    2534 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-token\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:11.416430 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:11.416443    2534 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8ea1f90-cf56-416b-bd20-79301599a9ea-istio-envoy\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:12.108016 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:12.107982    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h" event={"ID":"c8ea1f90-cf56-416b-bd20-79301599a9ea","Type":"ContainerDied","Data":"3ae18e92be0a0e7a55dfcd4d0374a6db93e624d1bc752dc8d1157906962ba874"}
Jul 26 16:29:12.108016 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:12.108006    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"
Jul 26 16:29:12.108528 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:12.108043    2534 scope.go:117] "RemoveContainer" containerID="1fc98c32c16098f7e0c0cbc2530d59c4f967fb2129c4e3e3668278962fb3adbe"
Jul 26 16:29:12.127910 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:12.127883    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"]
Jul 26 16:29:12.131724 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:12.131702    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-8c3dba-openshift-default-649cd6f8fd-hhm6h"]
Jul 26 16:29:13.294221 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:13.294187    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" path="/var/lib/kubelet/pods/c8ea1f90-cf56-416b-bd20-79301599a9ea/volumes"
Jul 26 16:29:19.401225 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:19.401190    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:29:28.602079 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602042    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"]
Jul 26 16:29:28.602489 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602414    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" containerName="payload-pre-processing"
Jul 26 16:29:28.602489 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602426    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" containerName="payload-pre-processing"
Jul 26 16:29:28.602489 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602442    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy"
Jul 26 16:29:28.602489 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602449    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy"
Jul 26 16:29:28.602489 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602457    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a861524e-a8fb-4bb9-afd6-98db8828f099" containerName="revoke-keys"
Jul 26 16:29:28.602489 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602463    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="a861524e-a8fb-4bb9-afd6-98db8828f099" containerName="revoke-keys"
Jul 26 16:29:28.602706 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602527    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy"
Jul 26 16:29:28.602706 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602535    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="a861524e-a8fb-4bb9-afd6-98db8828f099" containerName="revoke-keys"
Jul 26 16:29:28.602706 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602542    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy"
Jul 26 16:29:28.602706 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602551    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="77ed2e71-e3b0-4544-a014-7ef3dffc2e5c" containerName="istio-proxy"
Jul 26 16:29:28.602706 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.602559    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd37c9f-218b-439d-b7a0-2ed0bec0b3b1" containerName="payload-pre-processing"
Jul 26 16:29:28.607317 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.607299    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:28.609572 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.609553    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 16:29:28.609678 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.609592    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-n79w9\""
Jul 26 16:29:28.612987 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.612964    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"]
Jul 26 16:29:28.681194 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.681159    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e6fb86-026d-4861-92e6-11898a88ac3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg\" (UID: \"30e6fb86-026d-4861-92e6-11898a88ac3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:28.782106 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.782072    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e6fb86-026d-4861-92e6-11898a88ac3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg\" (UID: \"30e6fb86-026d-4861-92e6-11898a88ac3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:28.782707 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.782690    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e6fb86-026d-4861-92e6-11898a88ac3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg\" (UID: \"30e6fb86-026d-4861-92e6-11898a88ac3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:28.919269 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:28.919166    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:29.049982 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:29.049956    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"]
Jul 26 16:29:29.051579 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:29:29.051544    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e6fb86_026d_4861_92e6_11898a88ac3c.slice/crio-6f974c4a6d339fe193e033e640a7b53c3ca8310f4872d6f70abedcfd1cdd5e16 WatchSource:0}: Error finding container 6f974c4a6d339fe193e033e640a7b53c3ca8310f4872d6f70abedcfd1cdd5e16: Status 404 returned error can't find the container with id 6f974c4a6d339fe193e033e640a7b53c3ca8310f4872d6f70abedcfd1cdd5e16
Jul 26 16:29:29.193922 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:29.193858    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg" event={"ID":"30e6fb86-026d-4861-92e6-11898a88ac3c","Type":"ContainerStarted","Data":"6f974c4a6d339fe193e033e640a7b53c3ca8310f4872d6f70abedcfd1cdd5e16"}
Jul 26 16:29:30.200574 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:30.200546    2534 generic.go:356] "Generic (PLEG): container finished" podID="30e6fb86-026d-4861-92e6-11898a88ac3c" containerID="ac44f8b385bde7a96b5d354a4c36a20df3d933de7f5de095a6a04e887e0690ea" exitCode=0
Jul 26 16:29:30.200955 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:30.200582    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg" event={"ID":"30e6fb86-026d-4861-92e6-11898a88ac3c","Type":"ContainerDied","Data":"ac44f8b385bde7a96b5d354a4c36a20df3d933de7f5de095a6a04e887e0690ea"}
Jul 26 16:29:31.333195 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:31.333169    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:31.407195 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:31.407152    2534 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e6fb86-026d-4861-92e6-11898a88ac3c-maas-api-service-ca\") pod \"30e6fb86-026d-4861-92e6-11898a88ac3c\" (UID: \"30e6fb86-026d-4861-92e6-11898a88ac3c\") "
Jul 26 16:29:31.407547 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:31.407524    2534 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30e6fb86-026d-4861-92e6-11898a88ac3c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "30e6fb86-026d-4861-92e6-11898a88ac3c" (UID: "30e6fb86-026d-4861-92e6-11898a88ac3c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 16:29:31.508760 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:31.508679    2534 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/30e6fb86-026d-4861-92e6-11898a88ac3c-maas-api-service-ca\") on node \"ip-10-0-138-109.ec2.internal\" DevicePath \"\""
Jul 26 16:29:32.211781 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:32.211752    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg" event={"ID":"30e6fb86-026d-4861-92e6-11898a88ac3c","Type":"ContainerDied","Data":"6f974c4a6d339fe193e033e640a7b53c3ca8310f4872d6f70abedcfd1cdd5e16"}
Jul 26 16:29:32.211941 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:32.211803    2534 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f974c4a6d339fe193e033e640a7b53c3ca8310f4872d6f70abedcfd1cdd5e16"
Jul 26 16:29:32.211941 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:32.211761    2534 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"
Jul 26 16:29:42.482852 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:42.482816    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-jhgzg"]
Jul 26 16:29:43.669457 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:43.669423    2534 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"]
Jul 26 16:29:43.670958 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:43.670935    2534 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-e0208c-ndtlg"]
Jul 26 16:29:45.293996 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:45.293961    2534 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30e6fb86-026d-4861-92e6-11898a88ac3c" path="/var/lib/kubelet/pods/30e6fb86-026d-4861-92e6-11898a88ac3c/volumes"
Jul 26 16:29:57.052027 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:57.051993    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-8nzs5_df18ad45-2d65-4175-9647-e4a654d79132/copy-manifests/0.log"
Jul 26 16:29:57.059914 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:57.059887    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-8nzs5_df18ad45-2d65-4175-9647-e4a654d79132/manager/0.log"
Jul 26 16:29:57.628477 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:57.628450    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751375-bpq48_81205460-7602-4ec5-be8a-029461f89b5d/cleanup/0.log"
Jul 26 16:29:59.335977 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:59.335945    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-29gdb_44c30aa3-d742-4a75-8cb9-b80cc92788d3/kuadrant-console-plugin/0.log"
Jul 26 16:29:59.688191 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:29:59.688098    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-jhgzg_920b7045-26d5-4c44-a1f6-625ee4fc2982/limitador/0.log"
Jul 26 16:30:00.528023 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:00.527992    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-mvv4m_c09f38e6-ee41-4c16-b61f-c4012f6a78ac/kube-auth-proxy/0.log"
Jul 26 16:30:01.455378 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:01.455347    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-69gvn_f4005f0d-d3a4-480b-ade6-16adbc2da7b5/main/0.log"
Jul 26 16:30:01.801516 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:01.801430    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-vwm4d_0f8b32a3-894e-4000-9bfa-4c2d442f0f63/main/0.log"
Jul 26 16:30:01.918312 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:01.918285    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-9np9g_f33625ef-d0ef-4d77-8618-6fba59b5a348/main/0.log"
Jul 26 16:30:10.772187 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:10.772154    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f5rzd_51eb5653-f3da-4900-8f1d-790a4ec88c7d/global-pull-secret-syncer/0.log"
Jul 26 16:30:10.991266 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:10.991225    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-t4kd8_40b953aa-b2b6-4ac4-a36b-94d60df89a5f/konnectivity-agent/0.log"
Jul 26 16:30:11.051117 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:11.051014    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-109.ec2.internal_a5682356d1d816bbe296f83312dcd1ed/haproxy/0.log"
Jul 26 16:30:16.242741 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:16.242711    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-29gdb_44c30aa3-d742-4a75-8cb9-b80cc92788d3/kuadrant-console-plugin/0.log"
Jul 26 16:30:16.607914 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:16.607832    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-jhgzg_920b7045-26d5-4c44-a1f6-625ee4fc2982/limitador/0.log"
Jul 26 16:30:18.370050 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.370014    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/alertmanager/0.log"
Jul 26 16:30:18.402473 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.402451    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/config-reloader/0.log"
Jul 26 16:30:18.430714 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.430690    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/kube-rbac-proxy-web/0.log"
Jul 26 16:30:18.472403 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.472378    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/kube-rbac-proxy/0.log"
Jul 26 16:30:18.503539 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.503507    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/kube-rbac-proxy-metric/0.log"
Jul 26 16:30:18.535091 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.535074    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/prom-label-proxy/0.log"
Jul 26 16:30:18.558605 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.558585    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_43a7dde5-457c-4b02-bbeb-ea8219908ffc/init-config-reloader/0.log"
Jul 26 16:30:18.664006 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.663938    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-2dp4s_a028f259-90e1-40ce-b2f2-9c54df73358a/kube-state-metrics/0.log"
Jul 26 16:30:18.683661 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.683635    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-2dp4s_a028f259-90e1-40ce-b2f2-9c54df73358a/kube-rbac-proxy-main/0.log"
Jul 26 16:30:18.710302 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.710283    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-2dp4s_a028f259-90e1-40ce-b2f2-9c54df73358a/kube-rbac-proxy-self/0.log"
Jul 26 16:30:18.744626 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.744587    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-868445f98d-q5ccn_f2fd3614-cc28-4a8d-9a64-a0318f058bdd/metrics-server/0.log"
Jul 26 16:30:18.775454 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.775432    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-8vcj4_f7f98ca9-91d9-4a94-a414-e75a90d9449a/monitoring-plugin/0.log"
Jul 26 16:30:18.904445 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.904415    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5zgs_a4f8f6cb-3d21-447e-8d59-287225dfc24f/node-exporter/0.log"
Jul 26 16:30:18.927979 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.927911    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5zgs_a4f8f6cb-3d21-447e-8d59-287225dfc24f/kube-rbac-proxy/0.log"
Jul 26 16:30:18.953536 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:18.953508    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f5zgs_a4f8f6cb-3d21-447e-8d59-287225dfc24f/init-textfile/0.log"
Jul 26 16:30:19.084492 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.084466    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vs7wb_7f19d119-1bac-45d6-a580-8b5bb5a1ba80/kube-rbac-proxy-main/0.log"
Jul 26 16:30:19.115349 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.115323    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vs7wb_7f19d119-1bac-45d6-a580-8b5bb5a1ba80/kube-rbac-proxy-self/0.log"
Jul 26 16:30:19.144111 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.144088    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vs7wb_7f19d119-1bac-45d6-a580-8b5bb5a1ba80/openshift-state-metrics/0.log"
Jul 26 16:30:19.215066 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.215042    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/prometheus/0.log"
Jul 26 16:30:19.251090 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.251069    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/config-reloader/0.log"
Jul 26 16:30:19.282743 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.282721    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/thanos-sidecar/0.log"
Jul 26 16:30:19.310175 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.310156    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/kube-rbac-proxy-web/0.log"
Jul 26 16:30:19.335145 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.335123    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/kube-rbac-proxy/0.log"
Jul 26 16:30:19.362487 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.362469    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/kube-rbac-proxy-thanos/0.log"
Jul 26 16:30:19.392010 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.391992    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_15d65b7b-3e3f-48a3-ad77-b4aa86c994c0/init-config-reloader/0.log"
Jul 26 16:30:19.437334 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.437309    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-qzcl2_1650e062-7149-4008-9c55-c512f46a9606/prometheus-operator/0.log"
Jul 26 16:30:19.461550 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.461531    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-qzcl2_1650e062-7149-4008-9c55-c512f46a9606/kube-rbac-proxy/0.log"
Jul 26 16:30:19.540464 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.540403    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cbd46bd9c-g5bqr_ab6c163d-8b6d-453d-b48a-f51ee53ff14e/telemeter-client/0.log"
Jul 26 16:30:19.564852 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.564830    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cbd46bd9c-g5bqr_ab6c163d-8b6d-453d-b48a-f51ee53ff14e/reload/0.log"
Jul 26 16:30:19.587424 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.587402    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cbd46bd9c-g5bqr_ab6c163d-8b6d-453d-b48a-f51ee53ff14e/kube-rbac-proxy/0.log"
Jul 26 16:30:19.622018 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.622000    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d54bbcc9b-xq2n4_afa9b5dc-344e-4fa4-9690-d16ab6b9715b/thanos-query/0.log"
Jul 26 16:30:19.645995 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.645975    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d54bbcc9b-xq2n4_afa9b5dc-344e-4fa4-9690-d16ab6b9715b/kube-rbac-proxy-web/0.log"
Jul 26 16:30:19.669905 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.669888    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d54bbcc9b-xq2n4_afa9b5dc-344e-4fa4-9690-d16ab6b9715b/kube-rbac-proxy/0.log"
Jul 26 16:30:19.693011 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.692995    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d54bbcc9b-xq2n4_afa9b5dc-344e-4fa4-9690-d16ab6b9715b/prom-label-proxy/0.log"
Jul 26 16:30:19.719218 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.719194    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d54bbcc9b-xq2n4_afa9b5dc-344e-4fa4-9690-d16ab6b9715b/kube-rbac-proxy-rules/0.log"
Jul 26 16:30:19.750332 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:19.750274    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d54bbcc9b-xq2n4_afa9b5dc-344e-4fa4-9690-d16ab6b9715b/kube-rbac-proxy-metrics/0.log"
Jul 26 16:30:20.540549 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.540510    2534 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"]
Jul 26 16:30:20.541090 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.541069    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30e6fb86-026d-4861-92e6-11898a88ac3c" containerName="revoke-keys"
Jul 26 16:30:20.541181 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.541092    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="30e6fb86-026d-4861-92e6-11898a88ac3c" containerName="revoke-keys"
Jul 26 16:30:20.541181 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.541106    2534 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy"
Jul 26 16:30:20.541181 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.541114    2534 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8ea1f90-cf56-416b-bd20-79301599a9ea" containerName="istio-proxy"
Jul 26 16:30:20.541376 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.541217    2534 memory_manager.go:368] "RemoveStaleState removing state" podUID="30e6fb86-026d-4861-92e6-11898a88ac3c" containerName="revoke-keys"
Jul 26 16:30:20.544905 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.544883    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.547200 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.547181    2534 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4m86b\"/\"default-dockercfg-gbd2s\""
Jul 26 16:30:20.547200 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.547190    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4m86b\"/\"openshift-service-ca.crt\""
Jul 26 16:30:20.548012 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.547996    2534 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4m86b\"/\"kube-root-ca.crt\""
Jul 26 16:30:20.554576 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.554556    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"]
Jul 26 16:30:20.612499 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.612478    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-podres\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.612622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.612507    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-lib-modules\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.612622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.612544    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-sys\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.612622 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.612604    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzz4v\" (UniqueName: \"kubernetes.io/projected/39ee84a2-cd60-470e-948f-70f754992186-kube-api-access-lzz4v\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.612727 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.612645    2534 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-proc\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713010 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.712984    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-podres\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713120 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713019    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-lib-modules\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713120 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713057    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-sys\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713120 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713094    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzz4v\" (UniqueName: \"kubernetes.io/projected/39ee84a2-cd60-470e-948f-70f754992186-kube-api-access-lzz4v\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713282 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713124    2534 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-proc\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713282 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713143    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-podres\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713282 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713188    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-sys\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713282 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713225    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-proc\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.713282 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.713224    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/39ee84a2-cd60-470e-948f-70f754992186-lib-modules\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.721223 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.721197    2534 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzz4v\" (UniqueName: \"kubernetes.io/projected/39ee84a2-cd60-470e-948f-70f754992186-kube-api-access-lzz4v\") pod \"perf-node-gather-daemonset-svb74\" (UID: \"39ee84a2-cd60-470e-948f-70f754992186\") " pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.854626 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.854548    2534 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:20.994071 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:20.994047    2534 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"]
Jul 26 16:30:20.996130 ip-10-0-138-109 kubenswrapper[2534]: W0726 16:30:20.996103    2534 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod39ee84a2_cd60_470e_948f_70f754992186.slice/crio-4541aaeb41ac869d104889ff8d59507107b3ec2076f05ee55d608e3f152aa8c3 WatchSource:0}: Error finding container 4541aaeb41ac869d104889ff8d59507107b3ec2076f05ee55d608e3f152aa8c3: Status 404 returned error can't find the container with id 4541aaeb41ac869d104889ff8d59507107b3ec2076f05ee55d608e3f152aa8c3
Jul 26 16:30:21.450497 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:21.450457    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74" event={"ID":"39ee84a2-cd60-470e-948f-70f754992186","Type":"ContainerStarted","Data":"ee90f1b000954dfd5494d62ec1069cdba904a71fc2f61816f3cf3dc0c283f582"}
Jul 26 16:30:21.450667 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:21.450529    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:21.450667 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:21.450557    2534 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74" event={"ID":"39ee84a2-cd60-470e-948f-70f754992186","Type":"ContainerStarted","Data":"4541aaeb41ac869d104889ff8d59507107b3ec2076f05ee55d608e3f152aa8c3"}
Jul 26 16:30:21.466936 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:21.466890    2534 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74" podStartSLOduration=1.466875543 podStartE2EDuration="1.466875543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 16:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 16:30:21.46493909 +0000 UTC m=+3120.649005228" watchObservedRunningTime="2026-07-26 16:30:21.466875543 +0000 UTC m=+3120.650941681"
Jul 26 16:30:22.190173 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:22.190144    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bb45cc56f-nxg6q_c697675b-acd7-4a74-ac80-1c0c84346e7f/console/0.log"
Jul 26 16:30:23.954349 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:23.954318    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6tbt2_fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65/dns/0.log"
Jul 26 16:30:23.976450 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:23.976423    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6tbt2_fdd5a0fe-9e63-4f61-9b20-a5c4fa92ea65/kube-rbac-proxy/0.log"
Jul 26 16:30:24.052567 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:24.052539    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9szlv_d998ee4d-b750-48d8-8fbd-2e086b48e1c5/dns-node-resolver/0.log"
Jul 26 16:30:24.726315 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:24.726289    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d7l7j_5b3996da-64e2-474e-b2c7-1b255286357b/node-ca/0.log"
Jul 26 16:30:26.094349 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:26.094301    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-mvv4m_c09f38e6-ee41-4c16-b61f-c4012f6a78ac/kube-auth-proxy/0.log"
Jul 26 16:30:27.218378 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:27.218345    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kv5pj_d6fb0601-03c3-4da0-ac2b-0baa98477acc/serve-healthcheck-canary/0.log"
Jul 26 16:30:27.465407 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:27.465378    2534 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4m86b/perf-node-gather-daemonset-svb74"
Jul 26 16:30:27.908551 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:27.908523    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xmfz_514e1705-96ac-435a-b371-d2fbcfa26998/kube-rbac-proxy/0.log"
Jul 26 16:30:27.935390 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:27.935360    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xmfz_514e1705-96ac-435a-b371-d2fbcfa26998/exporter/0.log"
Jul 26 16:30:27.958744 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:27.958700    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8xmfz_514e1705-96ac-435a-b371-d2fbcfa26998/extractor/0.log"
Jul 26 16:30:31.506937 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:31.506909    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-8nzs5_df18ad45-2d65-4175-9647-e4a654d79132/manager/0.log"
Jul 26 16:30:31.533262 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:31.533225    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66d4685587-8nzs5_df18ad45-2d65-4175-9647-e4a654d79132/copy-manifests/0.log"
Jul 26 16:30:31.804545 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:31.804452    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-mvv4m_c09f38e6-ee41-4c16-b61f-c4012f6a78ac/kube-auth-proxy/0.log"
Jul 26 16:30:32.833640 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:32.833615    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54c98bf85c-q226v_f6b3291c-0027-488b-b524-4ec99718160d/manager/0.log"
Jul 26 16:30:33.665503 ip-10-0-138-109 kubenswrapper[2534]: I0726 16:30:33.665471    2534 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-694bf5b6d6-mvv4m_c09f38e6-ee41-4c16-b61f-c4012f6a78ac/kube-auth-proxy/0.log"