Jul 29 07:15:16.940798 ip-10-0-137-55 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 07:15:17.296176 ip-10-0-137-55 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 07:15:17.296176 ip-10-0-137-55 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 07:15:17.296176 ip-10-0-137-55 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 07:15:17.296176 ip-10-0-137-55 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 07:15:17.296176 ip-10-0-137-55 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 07:15:17.297937 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.297859    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 07:15:17.302606 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302585    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 07:15:17.302606 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302601    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 07:15:17.302606 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302604    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 07:15:17.302606 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302608    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 07:15:17.302606 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302611    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 07:15:17.302606 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302614    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302617    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302619    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302622    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302626    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302629    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302631    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302634    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302637    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302639    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302642    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302645    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302648    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302651    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302653    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302656    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302659    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302661    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302664    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302667    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 07:15:17.302823 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302669    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302672    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302679    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302681    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302684    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302686    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302689    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302693    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302697    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302700    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302703    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302705    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302708    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302711    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302716    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302720    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302723    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302726    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302728    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 07:15:17.303305 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302731    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302734    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302737    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302740    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302742    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302745    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302748    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302751    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302754    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302756    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302759    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302761    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302765    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302768    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302771    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302774    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302777    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302779    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302782    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 07:15:17.303942 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302785    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302787    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302790    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302793    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302795    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302798    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302801    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302803    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302807    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302810    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302813    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302815    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302818    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302820    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302823    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302825    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302828    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302831    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302834    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302837    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 07:15:17.304401 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302840    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302843    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302846    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302848    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302851    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302854    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302856    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302859    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302861    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302864    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302866    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302870    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302873    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302875    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302878    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.302880    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303587    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303595    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303598    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303601    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 07:15:17.304898 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303604    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303607    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303609    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303612    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303615    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303618    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303620    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303623    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303626    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303628    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303631    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303634    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303636    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303640    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303643    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303645    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303648    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303650    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303653    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303655    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 07:15:17.305380 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303658    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303660    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303663    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303666    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303669    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303672    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303674    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303677    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303679    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303682    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303684    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303687    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303689    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303692    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303694    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303697    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303700    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303703    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303705    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 07:15:17.305901 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303708    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303710    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303718    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303723    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303726    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303728    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303731    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303734    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303736    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303739    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303742    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303744    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303747    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303750    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303753    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303756    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303758    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303761    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303764    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303766    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 07:15:17.306373 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303769    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303774    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303777    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303780    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303783    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303786    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303789    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303792    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303794    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303797    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303800    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303802    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303805    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303808    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303810    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303813    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303816    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303818    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303823    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 07:15:17.306877 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303825    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303830    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303833    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303836    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303839    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303842    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303844    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303847    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303850    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303853    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303856    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303859    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303862    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303864    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303867    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303869    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.303872    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304242    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304248    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 07:15:17.307341 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304251    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304254    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304257    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304260    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304263    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304265    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304268    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304270    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304273    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304275    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304278    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304280    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304283    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304285    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304288    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304291    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304293    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304295    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304298    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304301    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 07:15:17.307851 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304304    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304306    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304309    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304312    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304314    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304317    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304320    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304322    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304325    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304329    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304332    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304335    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304337    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304340    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304342    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304345    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304347    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304350    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304353    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304355    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 07:15:17.308334 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304358    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304361    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304363    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304366    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304369    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304371    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304374    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304377    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304380    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304383    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304385    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304388    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304391    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304393    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304396    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304399    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304401    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304404    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304407    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304410    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 07:15:17.308831 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304412    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304415    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304418    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304420    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304423    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304427    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304430    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304432    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304436    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304440    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304443    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304457    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304461    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304463    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304466    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304469    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304479    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304483    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304485    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 07:15:17.309353 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304488    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304491    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304493    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304496    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304498    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304501    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304504    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304506    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304524    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304527    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304529    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304532    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304534    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304537    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304540    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304542    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304545    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.304548    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304612    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304619    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 07:15:17.309820 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304625    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304631    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304637    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304641    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304647    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304654    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304657    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304661    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304664    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304667    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304671    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304674    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304677    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304680    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304684    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304687    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304691    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304694    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304697    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304700    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304704    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304707    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304710    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 07:15:17.310347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304713    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304716    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304719    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304722    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304725    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304728    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304731    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304735    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304738    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304741    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304744    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304747    2577 flags.go:64] FLAG: --enable-server="true"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304749    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304753    2577 flags.go:64] FLAG: --event-burst="100"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304756    2577 flags.go:64] FLAG: --event-qps="50"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304759    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304762    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304765    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304769    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304772    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304775    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304778    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304781    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304783    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304787    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 07:15:17.310938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304790    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304793    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304796    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304798    2577 flags.go:64] FLAG: --feature-gates=""
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304802    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304805    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304808    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304811    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304814    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304817    2577 flags.go:64] FLAG: --help="false"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304820    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-137-55.ec2.internal"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304823    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304825    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304828    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304831    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304835    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304837    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304840    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304843    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304846    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304849    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304852    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304854    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304857    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 07:15:17.311561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304859    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304862    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304868    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304870    2577 flags.go:64] FLAG: --lock-file=""
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304873    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304876    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304879    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304884    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304887    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304890    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304893    2577 flags.go:64] FLAG: --logging-format="text"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304895    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304899    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304902    2577 flags.go:64] FLAG: --manifest-url=""
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304905    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304909    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304912    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304916    2577 flags.go:64] FLAG: --max-pods="110"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304920    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304923    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304926    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304929    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304932    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304935    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304938    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 07:15:17.312565 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304944    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304947    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304951    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304953    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304956    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304961    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304964    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304967    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304970    2577 flags.go:64] FLAG: --port="10250"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304974    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304977    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bfb29c878544297f"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304980    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304982    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304985    2577 flags.go:64] FLAG: --register-node="true"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304988    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304991    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304995    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.304997    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305000    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305004    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305006    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305009    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305012    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305015    2577 flags.go:64] FLAG: --runonce="false"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305017    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 07:15:17.313636 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305020    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305025    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305027    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305030    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305033    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305036    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305039    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305042    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305045    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305047    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305050    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305053    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305056    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305059    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305064    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305066    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305070    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305075    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305078    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305080    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305083    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305086    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305089    2577 flags.go:64] FLAG: --v="2"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305093    2577 flags.go:64] FLAG: --version="false"
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305098    2577 flags.go:64] FLAG: --vmodule=""
Jul 29 07:15:17.314493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305101    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.305104    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305220    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305223    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305226    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305229    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305232    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305234    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305238    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305240    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305243    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305245    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305248    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305251    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305253    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305257    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305261    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305263    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305266    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305269    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 07:15:17.315290 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305271    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305274    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305276    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305280    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305283    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305285    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305288    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305291    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305293    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305296    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305298    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305301    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305304    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305306    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305310    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305312    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305314    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305317    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305319    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305322    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 07:15:17.316287 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305327    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305330    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305332    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305335    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305337    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305340    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305342    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305345    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305347    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305350    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305353    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305356    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305358    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305361    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305363    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305367    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305370    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305372    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305376    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 07:15:17.316962 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305380    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305384    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305387    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305390    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305394    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305397    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305401    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305404    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305406    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305409    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305412    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305414    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305417    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305421    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305423    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305426    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305429    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305431    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305434    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 07:15:17.317545 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305436    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305439    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305441    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305444    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305447    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305449    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305452    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305455    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305457    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305461    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305464    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305466    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305469    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305471    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305474    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305476    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305479    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305482    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305484    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305487    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 07:15:17.318332 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305490    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305493    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.305495    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.306274    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.314152    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.314175    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.315003    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.315023    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315140    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315149    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315155    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315161    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315168    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315174    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 07:15:17.319065 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315179    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315184    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315188    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315193    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315197    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315201    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315205    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315209    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315213    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315217    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315222    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315226    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315230    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315235    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315239    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315243    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315247    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315251    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315255    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315260    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 07:15:17.319421 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315264    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315270    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315276    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315289    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315294    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315298    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315302    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315306    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315310    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315314    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315318    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315322    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315327    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315331    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315335    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315340    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315344    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315350    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315356    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 07:15:17.320103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315360    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315365    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315369    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315373    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315378    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315383    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315388    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315393    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315397    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315401    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315406    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315410    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315414    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315418    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315423    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315427    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315432    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315444    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315449    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 07:15:17.320625 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315453    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315457    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315461    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315466    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315470    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315474    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315478    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315482    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315487    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315491    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315495    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315500    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315505    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315523    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315528    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315532    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315536    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315539    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315543    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 07:15:17.321139 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315547    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315551    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315554    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315558    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315562    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315566    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315570    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315574    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315578    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315581    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315586    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315590    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315602    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315606    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315610    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 07:15:17.321657 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315614    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.315623    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315880    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315891    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315897    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315902    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315907    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315912    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315916    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315921    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315925    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315929    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315933    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315937    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315942    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 07:15:17.322097 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315945    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315950    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315954    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315958    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315962    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315967    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315971    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315975    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315979    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315983    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315987    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315991    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.315996    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316000    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316005    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316019    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316024    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316028    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316032    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316037    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 07:15:17.322499 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316041    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316045    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316049    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316053    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316058    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316062    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316066    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316070    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316074    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316078    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316083    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316087    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316091    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316095    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316100    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316104    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316108    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316112    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316117    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316121    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 07:15:17.323001 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316125    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316132    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316138    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316143    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316147    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316152    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316156    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316161    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316177    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316181    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316186    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316190    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316193    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316198    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316202    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316209    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316214    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316218    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316223    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316228    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 07:15:17.323838 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316232    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316236    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316241    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316245    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316249    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316253    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316257    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316262    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316266    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316270    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316274    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316278    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316282    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316286    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316290    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316294    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316298    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316302    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316306    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 07:15:17.324598 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316310    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316316    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316329    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316334    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316338    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316342    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:17.316346    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.316354    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.316538    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.319544    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.321024    2577 server.go:1019] "Starting client certificate rotation"
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.321130    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.321676    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 07:15:17.325200 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.323808    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 07:15:17.326211 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.326191    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 07:15:17.335002 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.334984    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 29 07:15:17.336940 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.336924    2577 log.go:25] "Validated CRI v1 image API"
Jul 29 07:15:17.338457 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.338439    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 07:15:17.340738 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.340717    2577 fs.go:135] Filesystem UUIDs: map[48ad7802-8096-4a0d-a900-3b21daad9af7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d01459b5-7c55-4842-a3cb-c9c2fe2c5025:/dev/nvme0n1p3]
Jul 29 07:15:17.340832 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.340736    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 07:15:17.346681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.346555    2577 manager.go:216] Machine: {Timestamp:2026-07-29 07:15:17.345293408 +0000 UTC m=+0.300748338 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109853 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec209a55aa18862b32be463e31506553 SystemUUID:ec209a55-aa18-862b-32be-463e31506553 BootID:8477679f-218c-4396-aabe-eb9c2f321cee Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a7:ae:81:6d:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a7:ae:81:6d:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:b9:7e:50:5f:fd Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 07:15:17.346681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.346669    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 07:15:17.346833 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.346815    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 07:15:17.349054 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.349029    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 07:15:17.349186 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.349058    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 07:15:17.350364 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.350352    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 07:15:17.350397 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.350367    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 07:15:17.350397 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.350376    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 07:15:17.350458 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.350407    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 07:15:17.351362 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.351351    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 07:15:17.353133 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353122    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 07:15:17.353270 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353260    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 07:15:17.353300 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353292    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 07:15:17.353330 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353305    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 07:15:17.353330 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353321    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 07:15:17.353330 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353329    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 29 07:15:17.353409 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353338    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 07:15:17.353699 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353687    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 07:15:17.353746 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353706    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 07:15:17.353998 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.353979    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 07:15:17.355468 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.355446    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 07:15:17.357599 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.357585    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 07:15:17.357655 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.357615    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 07:15:17.359204 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359185    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359211    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359220    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359225    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359234    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359239    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359247    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359252    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359259    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359264    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359272    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 07:15:17.359338 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.359278    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 07:15:17.360020 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.360007    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 07:15:17.360020 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.360019    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 07:15:17.362102 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.362079    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 07:15:17.362160 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.362094    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 07:15:17.362160 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.362131    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 07:15:17.363230 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.363218    2577 server.go:1268] "Started kubelet"
Jul 29 07:15:17.363310 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.363289    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 07:15:17.363870 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.363841    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 07:15:17.363947 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.363887    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 07:15:17.366090 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.366068    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 07:15:17.366748 ip-10-0-137-55 systemd[1]: Started Kubernetes Kubelet.
Jul 29 07:15:17.368732 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.368381    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 07:15:17.368732 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.368596    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 07:15:17.372128 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.371251    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-55.ec2.internal.18c6b15d7ba0bfe9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-55.ec2.internal,UID:ip-10-0-137-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-55.ec2.internal,},FirstTimestamp:2026-07-29 07:15:17.363187689 +0000 UTC m=+0.318642618,LastTimestamp:2026-07-29 07:15:17.363187689 +0000 UTC m=+0.318642618,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-55.ec2.internal,}"
Jul 29 07:15:17.375940 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.375922    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 07:15:17.376030 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.375939    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 07:15:17.376079 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.376061    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:17.376619 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.376598    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 07:15:17.376695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.376629    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 07:15:17.376695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.376641    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 07:15:17.376805 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.376718    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 07:15:17.376805 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.376728    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 07:15:17.377042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377026    2577 factory.go:55] Registering systemd factory
Jul 29 07:15:17.377042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377041    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 29 07:15:17.377242 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377228    2577 factory.go:153] Registering CRI-O factory
Jul 29 07:15:17.377306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377247    2577 factory.go:223] Registration of the crio container factory successfully
Jul 29 07:15:17.377306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377295    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 07:15:17.377405 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377320    2577 factory.go:103] Registering Raw factory
Jul 29 07:15:17.377405 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377334    2577 manager.go:1196] Started watching for new ooms in manager
Jul 29 07:15:17.377772 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.377755    2577 manager.go:349] Starting recovery of all containers
Jul 29 07:15:17.385935 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.385908    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 07:15:17.386069 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.385968    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 07:15:17.387866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.387851    2577 manager.go:354] Recovery completed
Jul 29 07:15:17.392268 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.392256    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:17.393490 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.393472    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tbdfw"
Jul 29 07:15:17.394295 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.394276    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:17.394395 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.394304    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:17.394395 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.394318    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:17.395562 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.395540    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:17.395562 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.395550    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 07:15:17.395692 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.395565    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 07:15:17.395692 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.395583    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 07:15:17.396291 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.396231    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-55.ec2.internal.18c6b15d7d7b610b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-55.ec2.internal,UID:ip-10-0-137-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-55.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-55.ec2.internal,},FirstTimestamp:2026-07-29 07:15:17.394293003 +0000 UTC m=+0.349747934,LastTimestamp:2026-07-29 07:15:17.394293003 +0000 UTC m=+0.349747934,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-55.ec2.internal,}"
Jul 29 07:15:17.396722 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.396697    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:17.396786 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.396727    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:17.396786 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.396737    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:17.397108 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.397095    2577 policy_none.go:49] "None policy: Start"
Jul 29 07:15:17.397146 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.397112    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 07:15:17.397146 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.397122    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 07:15:17.398035 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.398025    2577 policy_none.go:47] "Start"
Jul 29 07:15:17.403694 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.403679    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tbdfw"
Jul 29 07:15:17.405772 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.405701    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-55.ec2.internal.18c6b15d7d7ba535  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-55.ec2.internal,UID:ip-10-0-137-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-55.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-55.ec2.internal,},FirstTimestamp:2026-07-29 07:15:17.394310453 +0000 UTC m=+0.349765384,LastTimestamp:2026-07-29 07:15:17.394310453 +0000 UTC m=+0.349765384,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-55.ec2.internal,}"
Jul 29 07:15:17.409418 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.409323    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-55.ec2.internal.18c6b15d7d7bd75f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-55.ec2.internal,UID:ip-10-0-137-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-55.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-55.ec2.internal,},FirstTimestamp:2026-07-29 07:15:17.394323295 +0000 UTC m=+0.349778226,LastTimestamp:2026-07-29 07:15:17.394323295 +0000 UTC m=+0.349778226,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-55.ec2.internal,}"
Jul 29 07:15:17.436176 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436063    2577 manager.go:337] "Starting Device Plugin manager"
Jul 29 07:15:17.436279 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.436209    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 07:15:17.436279 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436222    2577 server.go:87] "Starting device plugin registration server"
Jul 29 07:15:17.436451 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436440    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 07:15:17.436520 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436453    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 07:15:17.436611 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436597    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 07:15:17.436669 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436658    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 07:15:17.436669 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.436668    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 07:15:17.437144 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.437108    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 07:15:17.437217 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.437176    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:17.536748 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.536693    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:17.537713 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.537627    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:17.537713 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.537662    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:17.537713 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.537680    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:17.537878 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.537718    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-55.ec2.internal"
Jul 29 07:15:17.549755 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.549694    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-55.ec2.internal"
Jul 29 07:15:17.549755 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.549717    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-55.ec2.internal\": node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:17.559554 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.559535    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:17.575159 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.575140    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 07:15:17.576322 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.576306    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 07:15:17.576382 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.576342    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 07:15:17.576382 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.576360    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 07:15:17.576469 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:17.576393    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 07:15:17.578148 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:17.578134    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 07:15:18.867979 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:18.867953    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:18.884135 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.884097    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"]
Jul 29 07:15:18.884242 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.884186    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:18.885187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.885170    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:18.885267 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.885202    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:18.885267 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.885213    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:18.886530 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.886518    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:18.886643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.886627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:18.886694 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.886661    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:18.887234 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.887217    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:18.887294 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.887247    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:18.887294 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.887263    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:18.887390 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.887264    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:18.887390 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.887350    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:18.887390 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.887369    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:18.888696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.888680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:18.888778 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.888708    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 07:15:18.889335 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.889320    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 07:15:18.889419 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.889345    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 07:15:18.889419 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.889355    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeHasSufficientPID"
Jul 29 07:15:18.914508 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:18.914487    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-55.ec2.internal\" not found" node="ip-10-0-137-55.ec2.internal"
Jul 29 07:15:18.918667 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:18.918651    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-55.ec2.internal\" not found" node="ip-10-0-137-55.ec2.internal"
Jul 29 07:15:18.968306 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:18.968288    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:18.986071 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.986044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a5b4731f07499e219250b8830f022c9a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal\" (UID: \"a5b4731f07499e219250b8830f022c9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:18.986136 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.986076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5b4731f07499e219250b8830f022c9a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal\" (UID: \"a5b4731f07499e219250b8830f022c9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:18.986136 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:18.986094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0b909cb70da34197f82620fdf4d9c81-config\") pod \"kube-apiserver-proxy-ip-10-0-137-55.ec2.internal\" (UID: \"b0b909cb70da34197f82620fdf4d9c81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.069431 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:19.069370    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:19.086760 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.086740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a5b4731f07499e219250b8830f022c9a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal\" (UID: \"a5b4731f07499e219250b8830f022c9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.086816 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.086773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a5b4731f07499e219250b8830f022c9a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal\" (UID: \"a5b4731f07499e219250b8830f022c9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.086816 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.086809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5b4731f07499e219250b8830f022c9a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal\" (UID: \"a5b4731f07499e219250b8830f022c9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.086880 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.086828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0b909cb70da34197f82620fdf4d9c81-config\") pod \"kube-apiserver-proxy-ip-10-0-137-55.ec2.internal\" (UID: \"b0b909cb70da34197f82620fdf4d9c81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.086880 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.086856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0b909cb70da34197f82620fdf4d9c81-config\") pod \"kube-apiserver-proxy-ip-10-0-137-55.ec2.internal\" (UID: \"b0b909cb70da34197f82620fdf4d9c81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.086955 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.086894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5b4731f07499e219250b8830f022c9a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal\" (UID: \"a5b4731f07499e219250b8830f022c9a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.169930 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:19.169868    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:19.218413 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.218371    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.222835 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.222702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.270013 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:19.269986    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:19.370467 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:19.370396    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:19.471056 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:19.471029    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:19.475150 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.475131    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 07:15:19.528258 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.528223    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 07:15:19.528733 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.528369    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 07:15:19.528733 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.528426    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 07:15:19.571358 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:19.571326    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-55.ec2.internal\" not found"
Jul 29 07:15:19.580745 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.580720    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 07:15:19.584295 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.584277    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.584347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.584294    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 07:15:19.596168 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.596149    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 07:15:19.597027 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.597009    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal"
Jul 29 07:15:19.599061 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.599042    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 07:15:19.604170 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.604154    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 07:15:19.615306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.615281    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 07:10:18 +0000 UTC" deadline="2027-12-26 01:34:12.063406002 +0000 UTC"
Jul 29 07:15:19.615385 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.615308    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12354h18m52.448102725s"
Jul 29 07:15:19.621102 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.621060    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5kwkr"
Jul 29 07:15:19.627849 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.627810    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5kwkr"
Jul 29 07:15:19.729789 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:19.729721    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b4731f07499e219250b8830f022c9a.slice/crio-c74ef4ae25b2871d5d2c616b8eb3a307f8eab8f4303155f149807a89013ad598 WatchSource:0}: Error finding container c74ef4ae25b2871d5d2c616b8eb3a307f8eab8f4303155f149807a89013ad598: Status 404 returned error can't find the container with id c74ef4ae25b2871d5d2c616b8eb3a307f8eab8f4303155f149807a89013ad598
Jul 29 07:15:19.730473 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:19.730447    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b909cb70da34197f82620fdf4d9c81.slice/crio-7ade6504962467025e76412c929447cc9670bfeff92cdd6a8761c317472a0475 WatchSource:0}: Error finding container 7ade6504962467025e76412c929447cc9670bfeff92cdd6a8761c317472a0475: Status 404 returned error can't find the container with id 7ade6504962467025e76412c929447cc9670bfeff92cdd6a8761c317472a0475
Jul 29 07:15:19.734511 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.734441    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 07:15:19.787016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.786959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal" event={"ID":"a5b4731f07499e219250b8830f022c9a","Type":"ContainerStarted","Data":"c74ef4ae25b2871d5d2c616b8eb3a307f8eab8f4303155f149807a89013ad598"}
Jul 29 07:15:19.787983 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.787961    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal" event={"ID":"b0b909cb70da34197f82620fdf4d9c81","Type":"ContainerStarted","Data":"7ade6504962467025e76412c929447cc9670bfeff92cdd6a8761c317472a0475"}
Jul 29 07:15:19.922793 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:19.922721    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 07:15:20.401268 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.401050    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 07:15:20.562637 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.562604    2577 apiserver.go:52] "Watching apiserver"
Jul 29 07:15:20.573284 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.573256    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 07:15:20.573706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.573676    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-bvg8l","openshift-multus/multus-gsnqt","openshift-network-operator/iptables-alerter-wkq6m","openshift-ovn-kubernetes/ovnkube-node-p7zx5","kube-system/global-pull-secret-syncer-ntlft","kube-system/konnectivity-agent-c29rm","kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qszhz","openshift-dns/node-resolver-825gr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal","openshift-multus/network-metrics-daemon-bdlx9","openshift-network-diagnostics/network-check-target-b6k65","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v","openshift-image-registry/node-ca-7x4dw"]
Jul 29 07:15:20.575328 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.575303    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.576641 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.576616    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.577810 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.577788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.578116 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.578093    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.578290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.578272    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 07:15:20.578368 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.578290    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6984b\""
Jul 29 07:15:20.578368 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.578304    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.578973 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.578957    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 07:15:20.579387 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.579356    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.579473 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.579414    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.579473 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.579429    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-m7sks\""
Jul 29 07:15:20.579574 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.579559    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 07:15:20.580220 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.579970    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.580603 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.580585    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 07:15:20.580755 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.580730    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fq6fw\""
Jul 29 07:15:20.581036 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.581020    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.581329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.581310    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.581414 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.581384    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:20.582252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.582177    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.582504 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.582489    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.583863 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.583499    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 07:15:20.583863 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.583751    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.584293 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.583976    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.584293 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.584150    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 07:15:20.584410 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.584330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 07:15:20.585157 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.585083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.585245 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.585217    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 07:15:20.585500 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.585484    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 07:15:20.585621 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.585602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.585796 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.585729    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fwv4d\""
Jul 29 07:15:20.586061 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.586043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.588280 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.588259    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-x7crp\""
Jul 29 07:15:20.589806 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.589763    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 07:15:20.591186 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591170    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.591414 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591400    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 07:15:20.591493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591464    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 07:15:20.591564 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591523    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.591564 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591532    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7q5d2\""
Jul 29 07:15:20.591660 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591569    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.591660 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591580    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fsr7d\""
Jul 29 07:15:20.591660 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.591540    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dxhbp\""
Jul 29 07:15:20.592132 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.592113    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.592386 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.592368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.594345 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-hostroot\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.594449 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-etc-kubernetes\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.594449 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0270a271-eec0-4a58-a6f1-a57996c15f95-dbus\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.594449 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.594449 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovnkube-config\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.594449 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysctl-d\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.594696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.594696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594492    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:20.594696 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.594567    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:20.594696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-socket-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594708    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lh5\" (UniqueName: \"kubernetes.io/projected/f5204748-189e-44c1-befa-f4cec2ca1aba-kube-api-access-s5lh5\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-cni-bin\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-systemd-units\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysctl-conf\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-system-cni-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-daemon-config\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.594883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-multus-certs\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b42c\" (UniqueName: \"kubernetes.io/projected/7f16983f-ba61-4f47-bed9-9d3a8f266eed-kube-api-access-5b42c\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b-agent-certs\") pod \"konnectivity-agent-c29rm\" (UID: \"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b\") " pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b-konnectivity-ca\") pod \"konnectivity-agent-c29rm\" (UID: \"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b\") " pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.594995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-kubelet\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovn-node-metrics-cert\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-conf-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595163    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mflh\" (UniqueName: \"kubernetes.io/projected/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-kube-api-access-6mflh\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-cni-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-cni-netd\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-etc-selinux\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-sys-fs\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.595290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-os-release\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-run-ovn-kubernetes\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595347    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f16983f-ba61-4f47-bed9-9d3a8f266eed-cni-binary-copy\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-ovn\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-modprobe-d\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-host\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-k8s-cni-cncf-io\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-slash\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595497    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-run-netns\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595523    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-sys\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92f62\" (UniqueName: \"kubernetes.io/projected/da39f9bb-6759-4a89-841e-a43eff6ceaa2-kube-api-access-92f62\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-kubernetes\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.595653    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-iptables-alerter-script\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.595990 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-os-release\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-socket-dir-parent\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-node-log\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-systemd\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0270a271-eec0-4a58-a6f1-a57996c15f95-kubelet-config\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595889    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-host-slash\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595942    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-system-cni-dir\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595968    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-registration-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.595993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-log-socket\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovnkube-script-lib\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596040    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-run\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596076    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-kubelet\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pzv6s\""
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596104    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-var-lib-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-tuned\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.596866 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596275    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-cni-bin\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-env-overrides\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-lib-modules\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596436    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cnibin\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-etc-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptpt2\" (UniqueName: \"kubernetes.io/projected/3f916b09-8e65-4147-a8d7-929eceb74dc4-kube-api-access-ptpt2\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-systemd\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596560    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da39f9bb-6759-4a89-841e-a43eff6ceaa2-tmp\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-device-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysconfig\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596656    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-var-lib-kubelet\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxjh\" (UniqueName: \"kubernetes.io/projected/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-kube-api-access-flxjh\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-cnibin\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-netns\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.597840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.596805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-cni-multus\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.629310 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.629260    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 07:10:19 +0000 UTC" deadline="2028-02-14 18:34:29.518855892 +0000 UTC"
Jul 29 07:15:20.629310 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.629298    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13571h19m8.889561327s"
Jul 29 07:15:20.685712 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.685686    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 07:15:20.697034 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t778f\" (UniqueName: \"kubernetes.io/projected/d9b974f2-1f07-4620-9199-4ff3c54c124f-kube-api-access-t778f\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.697166 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-ovn\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697166 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697087    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-modprobe-d\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697166 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-ovn\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697320 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-host\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697320 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-host\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697320 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-k8s-cni-cncf-io\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.697320 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697266    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-modprobe-d\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697320 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-slash\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697320 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-k8s-cni-cncf-io\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-run-netns\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697354    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-slash\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697370    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-sys\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92f62\" (UniqueName: \"kubernetes.io/projected/da39f9bb-6759-4a89-841e-a43eff6ceaa2-kube-api-access-92f62\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-run-netns\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-sys\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697420    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-kubernetes\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-iptables-alerter-script\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-os-release\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-socket-dir-parent\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-node-log\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-systemd\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0270a271-eec0-4a58-a6f1-a57996c15f95-kubelet-config\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.697600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-host-slash\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-system-cni-dir\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-registration-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697651    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-kubernetes\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-log-socket\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697708    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-os-release\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697616    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-log-socket\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovnkube-script-lib\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697915    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-run\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-kubelet\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697969    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-host-slash\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.697973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-var-lib-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-var-lib-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-socket-dir-parent\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-tuned\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.698400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0270a271-eec0-4a58-a6f1-a57996c15f95-kubelet-config\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698062    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-cni-bin\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-env-overrides\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-system-cni-dir\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-lib-modules\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cnibin\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9b974f2-1f07-4620-9199-4ff3c54c124f-host\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx5rf\" (UniqueName: \"kubernetes.io/projected/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-kube-api-access-tx5rf\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.698238    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-etc-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698275    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptpt2\" (UniqueName: \"kubernetes.io/projected/3f916b09-8e65-4147-a8d7-929eceb74dc4-kube-api-access-ptpt2\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-node-log\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698280    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.698323    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:15:21.19828098 +0000 UTC m=+2.946346844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-iptables-alerter-script\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-run\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.699202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-systemd\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da39f9bb-6759-4a89-841e-a43eff6ceaa2-tmp\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-device-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92cfc2e5-14c0-4b60-a417-60a688d08203-tmp-dir\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovnkube-script-lib\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysconfig\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698608    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-var-lib-kubelet\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flxjh\" (UniqueName: \"kubernetes.io/projected/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-kube-api-access-flxjh\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698640    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-lib-modules\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698638    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-kubelet\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-cnibin\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-netns\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysconfig\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-env-overrides\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-cni-bin\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cnibin\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.700019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698828    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-var-lib-kubelet\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-etc-openvswitch\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-cni-multus\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-netns\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-device-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-run-systemd\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-systemd\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698959    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-cni-multus\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.698990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-registration-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-hostroot\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699031    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-cnibin\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-etc-kubernetes\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0270a271-eec0-4a58-a6f1-a57996c15f95-dbus\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovnkube-config\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysctl-d\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.700822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-socket-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699150    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-etc-kubernetes\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699259    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0270a271-eec0-4a58-a6f1-a57996c15f95-dbus\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lh5\" (UniqueName: \"kubernetes.io/projected/f5204748-189e-44c1-befa-f4cec2ca1aba-kube-api-access-s5lh5\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699328    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-cni-bin\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-systemd-units\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysctl-conf\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-system-cni-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699394    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysctl-d\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-daemon-config\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-multus-certs\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b42c\" (UniqueName: \"kubernetes.io/projected/7f16983f-ba61-4f47-bed9-9d3a8f266eed-kube-api-access-5b42c\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92cfc2e5-14c0-4b60-a417-60a688d08203-hosts-file\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-hostroot\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.701615 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b-agent-certs\") pod \"konnectivity-agent-c29rm\" (UID: \"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b\") " pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b-konnectivity-ca\") pod \"konnectivity-agent-c29rm\" (UID: \"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b\") " pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-kubelet\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql4jh\" (UniqueName: \"kubernetes.io/projected/92cfc2e5-14c0-4b60-a417-60a688d08203-kube-api-access-ql4jh\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovn-node-metrics-cert\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-conf-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d9b974f2-1f07-4620-9199-4ff3c54c124f-serviceca\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mflh\" (UniqueName: \"kubernetes.io/projected/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-kube-api-access-6mflh\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699731    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovnkube-config\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-cni-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-cni-netd\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-etc-selinux\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699812    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-sys-fs\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-os-release\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-run-ovn-kubernetes\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.702417 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f16983f-ba61-4f47-bed9-9d3a8f266eed-cni-binary-copy\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699881    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-socket-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-daemon-config\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-systemd-units\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-cni-bin\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700120    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-system-cni-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-sysctl-conf\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-cni-netd\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-etc-selinux\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.699795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-cni-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-run-multus-certs\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-sys-fs\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f5204748-189e-44c1-befa-f4cec2ca1aba-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-os-release\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f916b09-8e65-4147-a8d7-929eceb74dc4-host-run-ovn-kubernetes\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-multus-conf-dir\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b-konnectivity-ca\") pod \"konnectivity-agent-c29rm\" (UID: \"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b\") " pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.703254 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.700609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f16983f-ba61-4f47-bed9-9d3a8f266eed-host-var-lib-kubelet\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.704138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.701001    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7f16983f-ba61-4f47-bed9-9d3a8f266eed-cni-binary-copy\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.704138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.701218    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.704138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.702547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da39f9bb-6759-4a89-841e-a43eff6ceaa2-tmp\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.704138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.702987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/da39f9bb-6759-4a89-841e-a43eff6ceaa2-etc-tuned\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.704138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.702999    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b-agent-certs\") pod \"konnectivity-agent-c29rm\" (UID: \"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b\") " pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.704138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.703021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f916b09-8e65-4147-a8d7-929eceb74dc4-ovn-node-metrics-cert\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.705874 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.705851    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92f62\" (UniqueName: \"kubernetes.io/projected/da39f9bb-6759-4a89-841e-a43eff6ceaa2-kube-api-access-92f62\") pod \"tuned-qszhz\" (UID: \"da39f9bb-6759-4a89-841e-a43eff6ceaa2\") " pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.709194 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.709175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxjh\" (UniqueName: \"kubernetes.io/projected/30337ef6-b0b7-4ca5-87bb-1074b9b2efef-kube-api-access-flxjh\") pod \"multus-additional-cni-plugins-bvg8l\" (UID: \"30337ef6-b0b7-4ca5-87bb-1074b9b2efef\") " pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.712139 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.712116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptpt2\" (UniqueName: \"kubernetes.io/projected/3f916b09-8e65-4147-a8d7-929eceb74dc4-kube-api-access-ptpt2\") pod \"ovnkube-node-p7zx5\" (UID: \"3f916b09-8e65-4147-a8d7-929eceb74dc4\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.713782 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.713763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b42c\" (UniqueName: \"kubernetes.io/projected/7f16983f-ba61-4f47-bed9-9d3a8f266eed-kube-api-access-5b42c\") pod \"multus-gsnqt\" (UID: \"7f16983f-ba61-4f47-bed9-9d3a8f266eed\") " pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.714883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.714859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mflh\" (UniqueName: \"kubernetes.io/projected/659ddb7b-63ef-4640-a3d2-9f8d0390ffd4-kube-api-access-6mflh\") pod \"iptables-alerter-wkq6m\" (UID: \"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4\") " pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.715449 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.715431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lh5\" (UniqueName: \"kubernetes.io/projected/f5204748-189e-44c1-befa-f4cec2ca1aba-kube-api-access-s5lh5\") pod \"aws-ebs-csi-driver-node-wl82v\" (UID: \"f5204748-189e-44c1-befa-f4cec2ca1aba\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.800642 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9b974f2-1f07-4620-9199-4ff3c54c124f-host\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.800642 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx5rf\" (UniqueName: \"kubernetes.io/projected/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-kube-api-access-tx5rf\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92cfc2e5-14c0-4b60-a417-60a688d08203-tmp-dir\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d9b974f2-1f07-4620-9199-4ff3c54c124f-host\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800757    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92cfc2e5-14c0-4b60-a417-60a688d08203-hosts-file\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql4jh\" (UniqueName: \"kubernetes.io/projected/92cfc2e5-14c0-4b60-a417-60a688d08203-kube-api-access-ql4jh\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d9b974f2-1f07-4620-9199-4ff3c54c124f-serviceca\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.800851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t778f\" (UniqueName: \"kubernetes.io/projected/d9b974f2-1f07-4620-9199-4ff3c54c124f-kube-api-access-t778f\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.801175 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.800890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/92cfc2e5-14c0-4b60-a417-60a688d08203-hosts-file\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.801175 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.800954    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:20.801175 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.801026    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:15:21.301006259 +0000 UTC m=+3.049072087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:20.801175 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.801030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92cfc2e5-14c0-4b60-a417-60a688d08203-tmp-dir\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.801336 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.801316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d9b974f2-1f07-4620-9199-4ff3c54c124f-serviceca\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.812344 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.812318    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:20.812344 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.812342    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:20.812480 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.812357    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:20.812480 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:20.812426    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:15:21.312408469 +0000 UTC m=+3.060474292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:20.815542 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.815519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx5rf\" (UniqueName: \"kubernetes.io/projected/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-kube-api-access-tx5rf\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:20.816086 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.816066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql4jh\" (UniqueName: \"kubernetes.io/projected/92cfc2e5-14c0-4b60-a417-60a688d08203-kube-api-access-ql4jh\") pod \"node-resolver-825gr\" (UID: \"92cfc2e5-14c0-4b60-a417-60a688d08203\") " pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.816196 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.816151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t778f\" (UniqueName: \"kubernetes.io/projected/d9b974f2-1f07-4620-9199-4ff3c54c124f-kube-api-access-t778f\") pod \"node-ca-7x4dw\" (UID: \"d9b974f2-1f07-4620-9199-4ff3c54c124f\") " pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:20.889133 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.889103    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v"
Jul 29 07:15:20.900706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.900686    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:20.909480 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.909461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gsnqt"
Jul 29 07:15:20.922965 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.922950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wkq6m"
Jul 29 07:15:20.931535 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.931515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:20.936989 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.936962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvg8l"
Jul 29 07:15:20.942529 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.942509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qszhz"
Jul 29 07:15:20.950129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.950109    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-825gr"
Jul 29 07:15:20.954499 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:20.954478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7x4dw"
Jul 29 07:15:21.203658 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.203565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:21.203822 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.203691    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:21.203822 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.203760    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:15:22.203740284 +0000 UTC m=+3.951806116 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:21.243461 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.243430    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 07:15:21.304433 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.304396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:21.304587 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.304535    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:21.304659 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.304616    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:15:22.304584898 +0000 UTC m=+4.052650718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:21.405379 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.405343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:21.405551 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.405499    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:21.405551 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.405525    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:21.405551 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.405539    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:21.405790 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.405600    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:15:22.405580179 +0000 UTC m=+4.153646009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:21.630115 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.630034    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 07:10:19 +0000 UTC" deadline="2028-01-01 09:24:22.473296577 +0000 UTC"
Jul 29 07:15:21.630115 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.630078    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12506h9m0.843220438s"
Jul 29 07:15:21.657893 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.657851    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf37a9f39_8a38_4e4f_87c7_e835d1fb9f1b.slice/crio-cd14cd526380bd20c80ecb64cf1f9d7f77f91ad54537d6843e8755e5ede0d816 WatchSource:0}: Error finding container cd14cd526380bd20c80ecb64cf1f9d7f77f91ad54537d6843e8755e5ede0d816: Status 404 returned error can't find the container with id cd14cd526380bd20c80ecb64cf1f9d7f77f91ad54537d6843e8755e5ede0d816
Jul 29 07:15:21.659934 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.659895    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b974f2_1f07_4620_9199_4ff3c54c124f.slice/crio-0502526a30b3cebefd66302549ed93f37ab208aff2d5e533c9681233297597e7 WatchSource:0}: Error finding container 0502526a30b3cebefd66302549ed93f37ab208aff2d5e533c9681233297597e7: Status 404 returned error can't find the container with id 0502526a30b3cebefd66302549ed93f37ab208aff2d5e533c9681233297597e7
Jul 29 07:15:21.660719 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.660699    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30337ef6_b0b7_4ca5_87bb_1074b9b2efef.slice/crio-60b11e4ede754a434b3df1f62c61d6bbbce7539d8a66d14c2317f0c5184471a3 WatchSource:0}: Error finding container 60b11e4ede754a434b3df1f62c61d6bbbce7539d8a66d14c2317f0c5184471a3: Status 404 returned error can't find the container with id 60b11e4ede754a434b3df1f62c61d6bbbce7539d8a66d14c2317f0c5184471a3
Jul 29 07:15:21.664278 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.664255    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f916b09_8e65_4147_a8d7_929eceb74dc4.slice/crio-d6c2064e6639f863d0faee71e3f032c90a5a9457e21914f5f6372488dd35eef1 WatchSource:0}: Error finding container d6c2064e6639f863d0faee71e3f032c90a5a9457e21914f5f6372488dd35eef1: Status 404 returned error can't find the container with id d6c2064e6639f863d0faee71e3f032c90a5a9457e21914f5f6372488dd35eef1
Jul 29 07:15:21.664805 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.664787    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659ddb7b_63ef_4640_a3d2_9f8d0390ffd4.slice/crio-788fd9dfadf81aa089ddf218073467feb9e37762a5ea6822d7cb4a3602d37393 WatchSource:0}: Error finding container 788fd9dfadf81aa089ddf218073467feb9e37762a5ea6822d7cb4a3602d37393: Status 404 returned error can't find the container with id 788fd9dfadf81aa089ddf218073467feb9e37762a5ea6822d7cb4a3602d37393
Jul 29 07:15:21.666333 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.666045    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92cfc2e5_14c0_4b60_a417_60a688d08203.slice/crio-18299ba6005f92e97dfa7161238765dbbb198ec85bc989ebfe9917a9bfb5625b WatchSource:0}: Error finding container 18299ba6005f92e97dfa7161238765dbbb198ec85bc989ebfe9917a9bfb5625b: Status 404 returned error can't find the container with id 18299ba6005f92e97dfa7161238765dbbb198ec85bc989ebfe9917a9bfb5625b
Jul 29 07:15:21.666701 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:15:21.666639    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5204748_189e_44c1_befa_f4cec2ca1aba.slice/crio-72eca7a3280d841c6ecfdf0866af66ca1a1bff665575a4137e1a2373e8977381 WatchSource:0}: Error finding container 72eca7a3280d841c6ecfdf0866af66ca1a1bff665575a4137e1a2373e8977381: Status 404 returned error can't find the container with id 72eca7a3280d841c6ecfdf0866af66ca1a1bff665575a4137e1a2373e8977381
Jul 29 07:15:21.784810 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.784781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:21.784931 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.784815    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:21.784931 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.784889    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:21.785028 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:21.784978    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:21.793511 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.793483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal" event={"ID":"b0b909cb70da34197f82620fdf4d9c81","Type":"ContainerStarted","Data":"2c290158fb7179b58549c7923f060ef1f20182a0c1aaeca72132e835d974e26a"}
Jul 29 07:15:21.794530 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.794510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gsnqt" event={"ID":"7f16983f-ba61-4f47-bed9-9d3a8f266eed","Type":"ContainerStarted","Data":"b7b5d2480868ac9aecc9f2ef8223382ae41b4dcf78cd9ae74405c0290eab3d56"}
Jul 29 07:15:21.798849 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.798830    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wkq6m" event={"ID":"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4","Type":"ContainerStarted","Data":"788fd9dfadf81aa089ddf218073467feb9e37762a5ea6822d7cb4a3602d37393"}
Jul 29 07:15:21.800381 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.800362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v" event={"ID":"f5204748-189e-44c1-befa-f4cec2ca1aba","Type":"ContainerStarted","Data":"72eca7a3280d841c6ecfdf0866af66ca1a1bff665575a4137e1a2373e8977381"}
Jul 29 07:15:21.801348 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.801329    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qszhz" event={"ID":"da39f9bb-6759-4a89-841e-a43eff6ceaa2","Type":"ContainerStarted","Data":"1d447f1a96878ae14292cb82cf204e618d01a641dbb7f8d20fcbef105b8a4b2a"}
Jul 29 07:15:21.802893 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.802872    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-825gr" event={"ID":"92cfc2e5-14c0-4b60-a417-60a688d08203","Type":"ContainerStarted","Data":"18299ba6005f92e97dfa7161238765dbbb198ec85bc989ebfe9917a9bfb5625b"}
Jul 29 07:15:21.805076 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.805058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"d6c2064e6639f863d0faee71e3f032c90a5a9457e21914f5f6372488dd35eef1"}
Jul 29 07:15:21.806090 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.806071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerStarted","Data":"60b11e4ede754a434b3df1f62c61d6bbbce7539d8a66d14c2317f0c5184471a3"}
Jul 29 07:15:21.807741 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.807709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7x4dw" event={"ID":"d9b974f2-1f07-4620-9199-4ff3c54c124f","Type":"ContainerStarted","Data":"0502526a30b3cebefd66302549ed93f37ab208aff2d5e533c9681233297597e7"}
Jul 29 07:15:21.808517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.808469    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-55.ec2.internal" podStartSLOduration=2.808454598 podStartE2EDuration="2.808454598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:21.808139869 +0000 UTC m=+3.556205708" watchObservedRunningTime="2026-07-29 07:15:21.808454598 +0000 UTC m=+3.556520453"
Jul 29 07:15:21.808887 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:21.808870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c29rm" event={"ID":"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b","Type":"ContainerStarted","Data":"cd14cd526380bd20c80ecb64cf1f9d7f77f91ad54537d6843e8755e5ede0d816"}
Jul 29 07:15:22.209739 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:22.209650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:22.209883 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.209799    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:22.209883 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.209861    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:15:24.209845035 +0000 UTC m=+5.957910866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:22.310477 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:22.310428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:22.310701 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.310682    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:22.310776 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.310756    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:15:24.310735408 +0000 UTC m=+6.058801243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:22.411313 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:22.411263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:22.411488 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.411447    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:22.411488 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.411466    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:22.411488 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.411479    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:22.411661 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.411539    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:15:24.411522129 +0000 UTC m=+6.159587963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:22.786915 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:22.786870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:22.787504 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:22.787052    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:22.819999 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:22.819962    2577 generic.go:356] "Generic (PLEG): container finished" podID="a5b4731f07499e219250b8830f022c9a" containerID="1934d8d819e5f442842cfc19c4bc683113761c656c5ec264b8fd4cd06c4813ce" exitCode=0
Jul 29 07:15:22.821181 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:22.820140    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal" event={"ID":"a5b4731f07499e219250b8830f022c9a","Type":"ContainerDied","Data":"1934d8d819e5f442842cfc19c4bc683113761c656c5ec264b8fd4cd06c4813ce"}
Jul 29 07:15:23.784910 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:23.784862    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:23.785095 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:23.785013    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:23.785401 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:23.785377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:23.785516 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:23.785475    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:23.839316 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:23.839283    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal" event={"ID":"a5b4731f07499e219250b8830f022c9a","Type":"ContainerStarted","Data":"2cfe7f53b672228974d700c5e8709ef5bdb791c142a25c4da8fb46e2476a3064"}
Jul 29 07:15:24.225911 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:24.225809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:24.226081 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.226012    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:24.226143 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.226088    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:15:28.226067317 +0000 UTC m=+9.974133140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:24.327176 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:24.326493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:24.327176 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.326703    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:24.327176 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.326768    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:15:28.326749698 +0000 UTC m=+10.074815531 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:24.427705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:24.427663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:24.427886 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.427874    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:24.427988 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.427895    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:24.427988 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.427922    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:24.427988 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.427979    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:15:28.427961192 +0000 UTC m=+10.176027026 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:24.785036 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:24.785007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:24.785202 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:24.785147    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:25.784803 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:25.784699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:25.785316 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:25.784839    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:25.785316 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:25.784964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:25.785316 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:25.785084    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:26.784162 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:26.784129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:26.784330 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:26.784290    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:27.784562 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:27.784092    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:27.784562 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:27.784160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:27.784562 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:27.784265    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:27.784562 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:27.784515    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:28.262256 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:28.262173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:28.262425 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.262325    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:28.262425 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.262383    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:15:36.262368044 +0000 UTC m=+18.010433864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:28.363730 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:28.363133    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:28.363730 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.363284    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:28.363730 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.363366    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:15:36.363345031 +0000 UTC m=+18.111410868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:28.464571 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:28.464460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:28.464744 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.464655    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:28.464744 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.464679    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:28.464744 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.464692    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:28.464921 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.464749    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:15:36.464729838 +0000 UTC m=+18.212795676 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:28.786180 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:28.785691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:28.786180 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:28.785815    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:29.784537 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:29.784271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:29.784537 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:29.784271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:29.784537 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:29.784399    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:29.784537 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:29.784488    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:30.784993 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:30.784954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:30.785624 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:30.785101    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:31.784877 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:31.784835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:31.785050 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:31.784835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:31.785050 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:31.784995    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:31.785050 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:31.785017    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:32.784073 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:32.784035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:32.784236 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:32.784188    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:33.784656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:33.784619    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:33.784656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:33.784664    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:33.785169 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:33.784764    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:33.785169 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:33.784909    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:34.784031 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:34.783994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:34.784206 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:34.784119    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:35.784551 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:35.784523    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:35.785101 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:35.784642    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:35.785101 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:35.784694    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:35.785101 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:35.784787    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:36.325663 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:36.325627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:36.325830 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.325806    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:36.325895 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.325883    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:15:52.325863854 +0000 UTC m=+34.073929676 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:36.426649 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:36.426609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:36.426798 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.426733    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:36.426798 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.426788    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:15:52.426774798 +0000 UTC m=+34.174840618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:36.527426 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:36.527383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:36.527614 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.527577    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:36.527614 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.527606    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:36.527614 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.527617    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:36.527735 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.527681    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:15:52.527665108 +0000 UTC m=+34.275730931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:36.784179 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:36.784093    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:36.784335 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:36.784238    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:37.784180 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:37.784141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:37.784629 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:37.784272    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:37.784629 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:37.784342    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:37.784629 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:37.784473    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:38.785506 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:38.785469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:38.785970 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:38.785597    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:39.784391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:39.784350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:39.784580 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:39.784395    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:39.784580 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:39.784496    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:39.784685 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:39.784607    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:40.784941 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.784543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:40.785590 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:40.785036    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:40.872455 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.872375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gsnqt" event={"ID":"7f16983f-ba61-4f47-bed9-9d3a8f266eed","Type":"ContainerStarted","Data":"5656ab47797fcd3f51781030b81205c0a915e62ebb57e97ae2da6b5cbf208e6c"}
Jul 29 07:15:40.873726 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.873699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v" event={"ID":"f5204748-189e-44c1-befa-f4cec2ca1aba","Type":"ContainerStarted","Data":"4c71f7cd44779910176dfc5b30bdd29e5fa3a9e4a2f2aa3a5f49f90f425887c0"}
Jul 29 07:15:40.875025 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.875003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qszhz" event={"ID":"da39f9bb-6759-4a89-841e-a43eff6ceaa2","Type":"ContainerStarted","Data":"3dc9f83aeb40af95412d17bf591d24d8286b330442a28e6667d89b4d163635b6"}
Jul 29 07:15:40.876491 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.876470    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-825gr" event={"ID":"92cfc2e5-14c0-4b60-a417-60a688d08203","Type":"ContainerStarted","Data":"0856c5c046110731beb02b4978fa1a39598697f5990864b4b7dba41dd962902b"}
Jul 29 07:15:40.879143 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.879122    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"7db5649c5a44df6cabc201f070c9a30de728eb1c82a6eb784774138b909bd2d5"}
Jul 29 07:15:40.879253 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.879192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"05270ef7812cde39ab5b80727f6031e14b98ac6fc4bc19a998ec5a0f59c9c69f"}
Jul 29 07:15:40.879253 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.879243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"e00ddca72a3c6035ac2d243af6767b3c245fec87a56a474cad6f470adbf23ccf"}
Jul 29 07:15:40.879364 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.879293    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"c290998308fb4696ac713db0a3dad2f6ddcca14fc0d94c22b83f59057fda42f2"}
Jul 29 07:15:40.879364 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.879340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"0eb003acf455a67fd93be0a3aa8a095424fa87a18b9fbf672d261293b083e9d4"}
Jul 29 07:15:40.879460 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.879388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"e326f66f53054b7e86ab8cfe5c5240fd2129d08f349dd8e1d2f1fdbd99b8ef85"}
Jul 29 07:15:40.880597 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.880566    2577 generic.go:356] "Generic (PLEG): container finished" podID="30337ef6-b0b7-4ca5-87bb-1074b9b2efef" containerID="6d0092b80b97b3f88fee9224a68a1433730a6b4b597c15ef9776911dc4c807c8" exitCode=0
Jul 29 07:15:40.880689 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.880637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerDied","Data":"6d0092b80b97b3f88fee9224a68a1433730a6b4b597c15ef9776911dc4c807c8"}
Jul 29 07:15:40.882143 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.882123    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7x4dw" event={"ID":"d9b974f2-1f07-4620-9199-4ff3c54c124f","Type":"ContainerStarted","Data":"16759f2dbaeb4fd706e5be3402cc65b15e8a221a386d6777867982b33db26ccb"}
Jul 29 07:15:40.883761 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.883739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c29rm" event={"ID":"f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b","Type":"ContainerStarted","Data":"c6227ac812e5d1d2eeb64de0eda3e8cada56954d42b37d5ba4be4c7a1ed4e08a"}
Jul 29 07:15:40.897029 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.896990    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-55.ec2.internal" podStartSLOduration=21.896977412 podStartE2EDuration="21.896977412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:23.855889591 +0000 UTC m=+5.603955436" watchObservedRunningTime="2026-07-29 07:15:40.896977412 +0000 UTC m=+22.645043250"
Jul 29 07:15:40.897327 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.897301    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gsnqt" podStartSLOduration=4.538346397 podStartE2EDuration="22.897293879s" totalImagesPullingTime="18.358947482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:40.896431456 +0000 UTC m=+22.644497297" watchObservedRunningTime="2026-07-29 07:15:40.897293879 +0000 UTC m=+22.645359720"
Jul 29 07:15:40.985612 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:40.985549    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7x4dw" podStartSLOduration=4.668164688 podStartE2EDuration="22.98552999s" totalImagesPullingTime="18.317365302s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:40.963387375 +0000 UTC m=+22.711453217" watchObservedRunningTime="2026-07-29 07:15:40.98552999 +0000 UTC m=+22.733595835"
Jul 29 07:15:41.004813 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.004766    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qszhz" podStartSLOduration=4.698549972 podStartE2EDuration="23.004752127s" totalImagesPullingTime="18.306202155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:41.004748272 +0000 UTC m=+22.752814126" watchObservedRunningTime="2026-07-29 07:15:41.004752127 +0000 UTC m=+22.752817969"
Jul 29 07:15:41.005457 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.005427    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c29rm" podStartSLOduration=4.688588148 podStartE2EDuration="23.005416653s" totalImagesPullingTime="18.316828505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:40.985063287 +0000 UTC m=+22.733129132" watchObservedRunningTime="2026-07-29 07:15:41.005416653 +0000 UTC m=+22.753482494"
Jul 29 07:15:41.022172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.022121    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-825gr" podStartSLOduration=4.71388976 podStartE2EDuration="23.022104321s" totalImagesPullingTime="18.308214561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:41.021746422 +0000 UTC m=+22.769812291" watchObservedRunningTime="2026-07-29 07:15:41.022104321 +0000 UTC m=+22.770170164"
Jul 29 07:15:41.462760 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.462738    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 07:15:41.656690 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.656578    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T07:15:41.462754127Z","UUID":"bdfd29aa-6ea5-4870-b1bb-5ff62abd6e0f","Handler":null,"Name":"","Endpoint":""}
Jul 29 07:15:41.658233 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.658206    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 07:15:41.658373 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.658246    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 07:15:41.784721 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.784644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:41.784721 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.784692    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:41.785299 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:41.784829    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:41.785299 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:41.784966    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:41.888862 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.888465    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wkq6m" event={"ID":"659ddb7b-63ef-4640-a3d2-9f8d0390ffd4","Type":"ContainerStarted","Data":"cd25f99960240618098f5e36eb01b8368a194d128643daf3e74b9e11d5c951df"}
Jul 29 07:15:41.891570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:41.891445    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v" event={"ID":"f5204748-189e-44c1-befa-f4cec2ca1aba","Type":"ContainerStarted","Data":"cf4c6b4bc35343b3c94a202ba081d96020c85f4ce261af8e73060949139e4ec7"}
Jul 29 07:15:42.784996 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:42.784914    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:42.785440 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:42.785069    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:42.896114 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:42.896075    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v" event={"ID":"f5204748-189e-44c1-befa-f4cec2ca1aba","Type":"ContainerStarted","Data":"b7f15bdda53896b42f3c3b74811dfe5f1d714fd89865224966ebbc7d3d8b3035"}
Jul 29 07:15:42.918915 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:42.918848    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wl82v" podStartSLOduration=4.438720989 podStartE2EDuration="24.918829834s" totalImagesPullingTime="20.480108845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:42.918365881 +0000 UTC m=+24.666431723" watchObservedRunningTime="2026-07-29 07:15:42.918829834 +0000 UTC m=+24.666895677"
Jul 29 07:15:42.919246 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:42.919209    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wkq6m" podStartSLOduration=6.604373847 podStartE2EDuration="24.919196987s" totalImagesPullingTime="18.31482314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:41.906842575 +0000 UTC m=+23.654908417" watchObservedRunningTime="2026-07-29 07:15:42.919196987 +0000 UTC m=+24.667262832"
Jul 29 07:15:43.784762 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:43.784599    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:43.784996 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:43.784564    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:43.784996 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:43.784855    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:43.784996 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:43.784961    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:43.902223 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:43.902187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"72a98ac856db54a7ecf7ecfb39239a8e02d69eca9fa0c4b7242acd643f5cfe6d"}
Jul 29 07:15:44.045195 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:44.045120    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:44.045709 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:44.045684    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:44.784870 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:44.784841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:44.785088 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:44.784986    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:44.903965 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:44.903940    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:44.904502 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:44.904476    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c29rm"
Jul 29 07:15:45.784453 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.784281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:45.784646 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.784281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:45.784646 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:45.784523    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:45.784646 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:45.784591    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:45.909461 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.909427    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" event={"ID":"3f916b09-8e65-4147-a8d7-929eceb74dc4","Type":"ContainerStarted","Data":"833fd5aa333428d03407172ed6f091edca417a87d87e602b305361dd5587dddf"}
Jul 29 07:15:45.910234 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.909682    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:45.910234 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.909746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:45.911299 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.911270    2577 generic.go:356] "Generic (PLEG): container finished" podID="30337ef6-b0b7-4ca5-87bb-1074b9b2efef" containerID="1825c63479547272b8f34845db6464cd2738632c4bd02c6558fef6448cb6f27b" exitCode=0
Jul 29 07:15:45.911436 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.911364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerDied","Data":"1825c63479547272b8f34845db6464cd2738632c4bd02c6558fef6448cb6f27b"}
Jul 29 07:15:45.926295 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.926260    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:45.940447 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:45.940411    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5" podStartSLOduration=9.55591094 podStartE2EDuration="27.940398474s" totalImagesPullingTime="18.384487534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:45.940380239 +0000 UTC m=+27.688446091" watchObservedRunningTime="2026-07-29 07:15:45.940398474 +0000 UTC m=+27.688464312"
Jul 29 07:15:46.784225 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:46.784199    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:46.784365 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:46.784306    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:46.915946 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:46.915866    2577 generic.go:356] "Generic (PLEG): container finished" podID="30337ef6-b0b7-4ca5-87bb-1074b9b2efef" containerID="12ce32ac571859a8f19fcda240104aa9ed5da8b0978a3c7416cd38304f8804ce" exitCode=0
Jul 29 07:15:46.915946 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:46.915933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerDied","Data":"12ce32ac571859a8f19fcda240104aa9ed5da8b0978a3c7416cd38304f8804ce"}
Jul 29 07:15:46.916530 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:46.916496    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:46.931231 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:46.931208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:15:47.784172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:47.784146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:47.784289 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:47.784271    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:47.784589 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:47.784146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:47.784589 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:47.784564    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:47.923649 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:47.923624    2577 generic.go:356] "Generic (PLEG): container finished" podID="30337ef6-b0b7-4ca5-87bb-1074b9b2efef" containerID="addd18e166d0b95455b2bd3d73aade53de4f4a9c6da1930f5757c04fe8560a45" exitCode=0
Jul 29 07:15:47.924037 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:47.923706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerDied","Data":"addd18e166d0b95455b2bd3d73aade53de4f4a9c6da1930f5757c04fe8560a45"}
Jul 29 07:15:48.785167 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:48.785134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:48.785332 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:48.785232    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:49.784682 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:49.784638    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:49.784682 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:49.784679    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:49.785204 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:49.784787    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:49.785204 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:49.784880    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:50.784392 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:50.784187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:50.784609 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:50.784582    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:51.784741 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:51.784705    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:51.784741 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:51.784742    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:51.785200 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:51.784828    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:51.785200 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:51.784952    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:52.346448 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:52.346418    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:52.346605 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.346537    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:52.346605 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.346582    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret podName:0270a271-eec0-4a58-a6f1-a57996c15f95 nodeName:}" failed. No retries permitted until 2026-07-29 07:16:24.346569219 +0000 UTC m=+66.094635039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret") pod "global-pull-secret-syncer-ntlft" (UID: "0270a271-eec0-4a58-a6f1-a57996c15f95") : object "kube-system"/"original-pull-secret" not registered
Jul 29 07:15:52.446810 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:52.446786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:52.446942 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.446919    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:52.447031 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.446978    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs podName:a9088f8d-af8f-497b-b4ba-4ed4ab95035b nodeName:}" failed. No retries permitted until 2026-07-29 07:16:24.446961491 +0000 UTC m=+66.195027310 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs") pod "network-metrics-daemon-bdlx9" (UID: "a9088f8d-af8f-497b-b4ba-4ed4ab95035b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 07:15:52.547604 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:52.547576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:52.547774 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.547707    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 07:15:52.547774 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.547720    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 07:15:52.547774 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.547729    2577 projected.go:196] Error preparing data for projected volume kube-api-access-qsfwh for pod openshift-network-diagnostics/network-check-target-b6k65: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:52.547774 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.547771    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh podName:c2ecb5e1-7181-4170-ae25-06a19a7cf1ec nodeName:}" failed. No retries permitted until 2026-07-29 07:16:24.547758927 +0000 UTC m=+66.295824746 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qsfwh" (UniqueName: "kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh") pod "network-check-target-b6k65" (UID: "c2ecb5e1-7181-4170-ae25-06a19a7cf1ec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 07:15:52.784416 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:52.784335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:52.784576 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:52.784441    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:52.936179 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:52.936147    2577 generic.go:356] "Generic (PLEG): container finished" podID="30337ef6-b0b7-4ca5-87bb-1074b9b2efef" containerID="4dad1f05f2bb023876654c759ad67ecfa2c496bfb8e47f66e1872a03c7c17ea7" exitCode=0
Jul 29 07:15:52.936562 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:52.936208    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerDied","Data":"4dad1f05f2bb023876654c759ad67ecfa2c496bfb8e47f66e1872a03c7c17ea7"}
Jul 29 07:15:53.784083 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:53.784050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:53.784255 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:53.784150    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:53.784255 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:53.784204    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:53.784344 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:53.784294    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:53.941557 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:53.941514    2577 generic.go:356] "Generic (PLEG): container finished" podID="30337ef6-b0b7-4ca5-87bb-1074b9b2efef" containerID="b37bb4d144786e995d78471aaf963c0d1b663a5805490053cc3c25e6d3f86121" exitCode=0
Jul 29 07:15:53.941931 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:53.941577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerDied","Data":"b37bb4d144786e995d78471aaf963c0d1b663a5805490053cc3c25e6d3f86121"}
Jul 29 07:15:54.784551 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:54.784524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:54.784710 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:54.784627    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:54.946430 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:54.946395    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" event={"ID":"30337ef6-b0b7-4ca5-87bb-1074b9b2efef","Type":"ContainerStarted","Data":"eb643fc48269b65ab5226594f313efe014c4f2638fcb2882f2181db7ce9362fa"}
Jul 29 07:15:54.970591 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:54.970545    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bvg8l" podStartSLOduration=8.454259789 podStartE2EDuration="36.970532066s" totalImagesPullingTime="28.516272277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:15:54.968916121 +0000 UTC m=+36.716981955" watchObservedRunningTime="2026-07-29 07:15:54.970532066 +0000 UTC m=+36.718597917"
Jul 29 07:15:55.784057 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:55.784026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:55.784057 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:55.784062    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:55.784300 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:55.784160    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:55.784300 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:55.784277    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:56.787316 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:56.787285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:56.787741 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:56.787410    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:57.784803 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:57.784769    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:57.784994 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:57.784780    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:57.784994 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:57.784885    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:15:57.784994 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:57.784955    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:58.785378 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:58.785345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:15:58.785740 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:58.785433    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:15:59.784208 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:59.784173    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:15:59.784208 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:15:59.784213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:15:59.784408 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:59.784294    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:15:59.784446 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:15:59.784408    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:16:00.787491 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:00.787466    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:00.787859 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:00.787568    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:16:01.784883 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:01.784855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:01.785046 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:01.784855    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:01.785046 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:01.784966    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:16:01.785133 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:01.785041    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:16:02.784035 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:02.784003    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:02.784612 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:02.784137    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:16:03.476427 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:03.476395    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntlft"]
Jul 29 07:16:03.476555 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:03.476537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:03.476679 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:03.476655    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:16:03.479379 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:03.479327    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bdlx9"]
Jul 29 07:16:03.479504 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:03.479476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:03.479607 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:03.479581    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:16:03.480108 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:03.480084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6k65"]
Jul 29 07:16:03.480231 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:03.480188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:03.480292 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:03.480270    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:16:04.785014 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:04.784817    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:04.785650 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:04.784868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:04.785650 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:04.785095    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:16:04.785650 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:04.785170    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:16:05.784139 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:05.784100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:05.784340 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:05.784225    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntlft" podUID="0270a271-eec0-4a58-a6f1-a57996c15f95"
Jul 29 07:16:06.784738 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:06.784702    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:06.785156 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:06.784840    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bdlx9" podUID="a9088f8d-af8f-497b-b4ba-4ed4ab95035b"
Jul 29 07:16:06.785156 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:06.784887    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:06.785156 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:06.784990    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b6k65" podUID="c2ecb5e1-7181-4170-ae25-06a19a7cf1ec"
Jul 29 07:16:07.565225 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.565197    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-55.ec2.internal" event="NodeReady"
Jul 29 07:16:07.565333 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.565305    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 07:16:07.615249 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.615214    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cn4fx"]
Jul 29 07:16:07.637986 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.637963    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-b6nlg"]
Jul 29 07:16:07.638113 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.638088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.640791 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.640767    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 07:16:07.640924 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.640767    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6scvq\""
Jul 29 07:16:07.640924 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.640767    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 07:16:07.641049 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.640954    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 07:16:07.651705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.651684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccd9d\" (UniqueName: \"kubernetes.io/projected/58ecc536-8a5c-42d1-8180-31d417acfa9e-kube-api-access-ccd9d\") pod \"ingress-canary-cn4fx\" (UID: \"58ecc536-8a5c-42d1-8180-31d417acfa9e\") " pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.651824 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.651765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ecc536-8a5c-42d1-8180-31d417acfa9e-cert\") pod \"ingress-canary-cn4fx\" (UID: \"58ecc536-8a5c-42d1-8180-31d417acfa9e\") " pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.651941 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.651923    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cn4fx"]
Jul 29 07:16:07.651996 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.651951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b6nlg"]
Jul 29 07:16:07.652046 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.652034    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.652095 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.652046    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bdhzd"]
Jul 29 07:16:07.654868 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.654851    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zj4vm\""
Jul 29 07:16:07.654987 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.654892    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 07:16:07.654987 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.654938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 07:16:07.655122 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.655059    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 07:16:07.655208 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.655194    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 07:16:07.669947 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.669923    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bdhzd"]
Jul 29 07:16:07.670045 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.669963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.672705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.672691    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6q689\""
Jul 29 07:16:07.672786 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.672738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 07:16:07.673074 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.673058    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 07:16:07.752508 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a60808d-8e2c-4192-ae4e-7db2f04c3306-tmp-dir\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.752508 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbv5z\" (UniqueName: \"kubernetes.io/projected/1a60808d-8e2c-4192-ae4e-7db2f04c3306-kube-api-access-lbv5z\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.752508 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752489    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.752706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9www5\" (UniqueName: \"kubernetes.io/projected/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-kube-api-access-9www5\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.752706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccd9d\" (UniqueName: \"kubernetes.io/projected/58ecc536-8a5c-42d1-8180-31d417acfa9e-kube-api-access-ccd9d\") pod \"ingress-canary-cn4fx\" (UID: \"58ecc536-8a5c-42d1-8180-31d417acfa9e\") " pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.752706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.752706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a60808d-8e2c-4192-ae4e-7db2f04c3306-config-volume\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.752706 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-data-volume\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.752882 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-crio-socket\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.752882 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ecc536-8a5c-42d1-8180-31d417acfa9e-cert\") pod \"ingress-canary-cn4fx\" (UID: \"58ecc536-8a5c-42d1-8180-31d417acfa9e\") " pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.752882 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.752770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a60808d-8e2c-4192-ae4e-7db2f04c3306-metrics-tls\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.756834 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.756818    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58ecc536-8a5c-42d1-8180-31d417acfa9e-cert\") pod \"ingress-canary-cn4fx\" (UID: \"58ecc536-8a5c-42d1-8180-31d417acfa9e\") " pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.761978 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.761955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccd9d\" (UniqueName: \"kubernetes.io/projected/58ecc536-8a5c-42d1-8180-31d417acfa9e-kube-api-access-ccd9d\") pod \"ingress-canary-cn4fx\" (UID: \"58ecc536-8a5c-42d1-8180-31d417acfa9e\") " pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.784353 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.784329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:07.787357 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.787342    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 07:16:07.853737 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a60808d-8e2c-4192-ae4e-7db2f04c3306-tmp-dir\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.853737 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbv5z\" (UniqueName: \"kubernetes.io/projected/1a60808d-8e2c-4192-ae4e-7db2f04c3306-kube-api-access-lbv5z\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9www5\" (UniqueName: \"kubernetes.io/projected/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-kube-api-access-9www5\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a60808d-8e2c-4192-ae4e-7db2f04c3306-config-volume\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-data-volume\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-crio-socket\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.853931    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a60808d-8e2c-4192-ae4e-7db2f04c3306-metrics-tls\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.854355 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.854117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a60808d-8e2c-4192-ae4e-7db2f04c3306-tmp-dir\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.854355 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.854320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-crio-socket\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854458 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.854351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-data-volume\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.854458 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.854437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1a60808d-8e2c-4192-ae4e-7db2f04c3306-config-volume\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.854545 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.854462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.856241 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.856220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1a60808d-8e2c-4192-ae4e-7db2f04c3306-metrics-tls\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.856332 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.856264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.862063 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.862044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9www5\" (UniqueName: \"kubernetes.io/projected/2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0-kube-api-access-9www5\") pod \"insights-runtime-extractor-b6nlg\" (UID: \"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0\") " pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.862394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.862378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbv5z\" (UniqueName: \"kubernetes.io/projected/1a60808d-8e2c-4192-ae4e-7db2f04c3306-kube-api-access-lbv5z\") pod \"dns-default-bdhzd\" (UID: \"1a60808d-8e2c-4192-ae4e-7db2f04c3306\") " pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:07.947454 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.947403    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cn4fx"
Jul 29 07:16:07.959875 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.959835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-b6nlg"
Jul 29 07:16:07.977497 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:07.977476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:08.187235 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.187195    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-b6nlg"]
Jul 29 07:16:08.190192 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.190168    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cn4fx"]
Jul 29 07:16:08.191683 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:08.191658    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab292ff_afd8_4b2e_b1eb_bfc61f5ff3e0.slice/crio-6f336782ed442c9e5bde52b91bb81a97ddf9c7282d3a1cb43a872e300cdd9ecf WatchSource:0}: Error finding container 6f336782ed442c9e5bde52b91bb81a97ddf9c7282d3a1cb43a872e300cdd9ecf: Status 404 returned error can't find the container with id 6f336782ed442c9e5bde52b91bb81a97ddf9c7282d3a1cb43a872e300cdd9ecf
Jul 29 07:16:08.193884 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:08.193859    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ecc536_8a5c_42d1_8180_31d417acfa9e.slice/crio-cf0f6f96aafb4eb2b2f45c0ca7e5fe1011b80e638f3978e9dda8c49e828d6b17 WatchSource:0}: Error finding container cf0f6f96aafb4eb2b2f45c0ca7e5fe1011b80e638f3978e9dda8c49e828d6b17: Status 404 returned error can't find the container with id cf0f6f96aafb4eb2b2f45c0ca7e5fe1011b80e638f3978e9dda8c49e828d6b17
Jul 29 07:16:08.197917 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.197882    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bdhzd"]
Jul 29 07:16:08.200931 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:08.200915    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a60808d_8e2c_4192_ae4e_7db2f04c3306.slice/crio-53032e5f24aa1e5a2634e79df565c4c45e0b9d89a7f289653c001d64728bb7d8 WatchSource:0}: Error finding container 53032e5f24aa1e5a2634e79df565c4c45e0b9d89a7f289653c001d64728bb7d8: Status 404 returned error can't find the container with id 53032e5f24aa1e5a2634e79df565c4c45e0b9d89a7f289653c001d64728bb7d8
Jul 29 07:16:08.785781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.785749    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:08.785999 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.785972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:08.790311 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.790284    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fb4zn\""
Jul 29 07:16:08.790695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.790515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 07:16:08.790695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.790562    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 07:16:08.790695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.790515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rkrh4\""
Jul 29 07:16:08.790847 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.790731    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 07:16:08.984342 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.984073    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cn4fx" event={"ID":"58ecc536-8a5c-42d1-8180-31d417acfa9e","Type":"ContainerStarted","Data":"cf0f6f96aafb4eb2b2f45c0ca7e5fe1011b80e638f3978e9dda8c49e828d6b17"}
Jul 29 07:16:08.986947 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.986916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdhzd" event={"ID":"1a60808d-8e2c-4192-ae4e-7db2f04c3306","Type":"ContainerStarted","Data":"53032e5f24aa1e5a2634e79df565c4c45e0b9d89a7f289653c001d64728bb7d8"}
Jul 29 07:16:08.988569 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:08.988547    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6nlg" event={"ID":"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0","Type":"ContainerStarted","Data":"6f336782ed442c9e5bde52b91bb81a97ddf9c7282d3a1cb43a872e300cdd9ecf"}
Jul 29 07:16:09.770577 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.769751    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"]
Jul 29 07:16:09.790942 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.790916    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"]
Jul 29 07:16:09.791380 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.790875    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:09.794067 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.794044    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 07:16:09.794186 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.794078    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mtkh5\""
Jul 29 07:16:09.872003 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.871962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caabdffe-32e1-4e3e-b216-e4abb7e24c50-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nxqjx\" (UID: \"caabdffe-32e1-4e3e-b216-e4abb7e24c50\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:09.973757 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:09.973341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caabdffe-32e1-4e3e-b216-e4abb7e24c50-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nxqjx\" (UID: \"caabdffe-32e1-4e3e-b216-e4abb7e24c50\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:09.973757 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:09.973536    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 07:16:09.973757 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:09.973599    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caabdffe-32e1-4e3e-b216-e4abb7e24c50-tls-certificates podName:caabdffe-32e1-4e3e-b216-e4abb7e24c50 nodeName:}" failed. No retries permitted until 2026-07-29 07:16:10.473580508 +0000 UTC m=+52.221646342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/caabdffe-32e1-4e3e-b216-e4abb7e24c50-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-nxqjx" (UID: "caabdffe-32e1-4e3e-b216-e4abb7e24c50") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 07:16:10.387382 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.387351    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fcd6844fc-lg5m6"]
Jul 29 07:16:10.409113 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.409075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fcd6844fc-lg5m6"]
Jul 29 07:16:10.409253 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.409193    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.412078 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.411995    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 07:16:10.412078 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.412081    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 07:16:10.412274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.412103    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 07:16:10.412274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.412124    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 07:16:10.413492 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.413401    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-26zrp\""
Jul 29 07:16:10.413492 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.413463    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 07:16:10.413492 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.413489    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 07:16:10.413715 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.413464    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 07:16:10.477438 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-serving-cert\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.477643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-oauth-config\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.477643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9trj\" (UniqueName: \"kubernetes.io/projected/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-kube-api-access-g9trj\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.477643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-service-ca\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.477643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477622    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-config\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.477855 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caabdffe-32e1-4e3e-b216-e4abb7e24c50-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nxqjx\" (UID: \"caabdffe-32e1-4e3e-b216-e4abb7e24c50\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:10.477855 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.477692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-oauth-serving-cert\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.481490 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.481465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/caabdffe-32e1-4e3e-b216-e4abb7e24c50-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-nxqjx\" (UID: \"caabdffe-32e1-4e3e-b216-e4abb7e24c50\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:10.578431 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.578393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9trj\" (UniqueName: \"kubernetes.io/projected/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-kube-api-access-g9trj\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.578591 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.578444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-service-ca\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.578591 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.578505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-config\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.578591 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.578535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-oauth-serving-cert\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.578721 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.578594    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-serving-cert\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.578721 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.578617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-oauth-config\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.579510 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.579462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-config\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.579624 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.579549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-service-ca\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.579624 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.579549    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-oauth-serving-cert\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.581676 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.581646    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-serving-cert\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.582440 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.582420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-oauth-config\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.587959 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.587937    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9trj\" (UniqueName: \"kubernetes.io/projected/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-kube-api-access-g9trj\") pod \"console-fcd6844fc-lg5m6\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") " pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:10.703089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.702957    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:10.721164 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:10.721132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:13.789467 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:13.789360    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"]
Jul 29 07:16:14.004578 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:14.004506    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaabdffe_32e1_4e3e_b216_e4abb7e24c50.slice/crio-82b029ae7830dd27430684d5044c58cc8d167c6fa976ab4d589acc7b27630902 WatchSource:0}: Error finding container 82b029ae7830dd27430684d5044c58cc8d167c6fa976ab4d589acc7b27630902: Status 404 returned error can't find the container with id 82b029ae7830dd27430684d5044c58cc8d167c6fa976ab4d589acc7b27630902
Jul 29 07:16:14.136190 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:14.136158    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fcd6844fc-lg5m6"]
Jul 29 07:16:14.145144 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:14.145110    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f0a7571_4fa1_44c0_a0d0_6616b5a7cdf0.slice/crio-ea9e07b8278f3b98f637a8357502fbe5f87cec7f9c6c0094088dcef74dc39644 WatchSource:0}: Error finding container ea9e07b8278f3b98f637a8357502fbe5f87cec7f9c6c0094088dcef74dc39644: Status 404 returned error can't find the container with id ea9e07b8278f3b98f637a8357502fbe5f87cec7f9c6c0094088dcef74dc39644
Jul 29 07:16:15.010406 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.010367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdhzd" event={"ID":"1a60808d-8e2c-4192-ae4e-7db2f04c3306","Type":"ContainerStarted","Data":"cfb40cbaec87f813af40c6678759e0cdba229e1baa5b427e49f47c07dd7ac741"}
Jul 29 07:16:15.010876 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.010435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bdhzd" event={"ID":"1a60808d-8e2c-4192-ae4e-7db2f04c3306","Type":"ContainerStarted","Data":"95335a14d820c004f2e8f552af3661247820c5a5d40317f3802c672b4c901515"}
Jul 29 07:16:15.011291 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.011270    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:15.014211 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.014165    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fcd6844fc-lg5m6" event={"ID":"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0","Type":"ContainerStarted","Data":"ea9e07b8278f3b98f637a8357502fbe5f87cec7f9c6c0094088dcef74dc39644"}
Jul 29 07:16:15.018101 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.018072    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6nlg" event={"ID":"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0","Type":"ContainerStarted","Data":"39f827b77ff088073d7d855ae36ddd92a0a0de6d83f10e737366138019b32478"}
Jul 29 07:16:15.020720 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.020696    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx" event={"ID":"caabdffe-32e1-4e3e-b216-e4abb7e24c50","Type":"ContainerStarted","Data":"82b029ae7830dd27430684d5044c58cc8d167c6fa976ab4d589acc7b27630902"}
Jul 29 07:16:15.024226 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.023668    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cn4fx" event={"ID":"58ecc536-8a5c-42d1-8180-31d417acfa9e","Type":"ContainerStarted","Data":"c8bf08ac981df3b492990f0c1eec9209c45d2bc0194b3deb37ba552b2012c3fa"}
Jul 29 07:16:15.035265 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.034661    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bdhzd" podStartSLOduration=2.875952011 podStartE2EDuration="8.03464128s" totalImagesPullingTime="5.158689269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:15.034210014 +0000 UTC m=+56.782275856" watchObservedRunningTime="2026-07-29 07:16:15.03464128 +0000 UTC m=+56.782707123"
Jul 29 07:16:15.055891 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:15.055680    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cn4fx" podStartSLOduration=2.246439735 podStartE2EDuration="8.055657914s" totalImagesPullingTime="5.809218179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:15.054763484 +0000 UTC m=+56.802829327" watchObservedRunningTime="2026-07-29 07:16:15.055657914 +0000 UTC m=+56.803723757"
Jul 29 07:16:16.029631 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.029563    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx" event={"ID":"caabdffe-32e1-4e3e-b216-e4abb7e24c50","Type":"ContainerStarted","Data":"981cd6da58cce14179de53416653f5c29d3046a0f6a49ade1014cd269852f13d"}
Jul 29 07:16:16.030176 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.030148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:16.035414 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.035388    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx"
Jul 29 07:16:16.078499 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.078325    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-nxqjx" podStartSLOduration=5.510745539 podStartE2EDuration="7.078305395s" totalImagesPullingTime="1.567559856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:16.050442194 +0000 UTC m=+57.798508040" watchObservedRunningTime="2026-07-29 07:16:16.078305395 +0000 UTC m=+57.826371238"
Jul 29 07:16:16.854396 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.854356    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h49x7"]
Jul 29 07:16:16.861313 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.861284    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:16.865792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.865524    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 07:16:16.865792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.865545    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 07:16:16.865792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.865586    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 07:16:16.865792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.865570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dvj4t\""
Jul 29 07:16:16.865792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.865532    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 07:16:16.865792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.865590    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 07:16:16.870549 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.870491    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h49x7"]
Jul 29 07:16:16.933028 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.933001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:16.933175 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.933064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80a0c65c-352f-4abb-af52-a597cc6457df-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:16.933175 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.933088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq25t\" (UniqueName: \"kubernetes.io/projected/80a0c65c-352f-4abb-af52-a597cc6457df-kube-api-access-zq25t\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:16.933175 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:16.933111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.033424 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.033386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.033862 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.033461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.033862 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.033521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80a0c65c-352f-4abb-af52-a597cc6457df-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.033862 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:17.033532    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 29 07:16:17.033862 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:17.033603    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-tls podName:80a0c65c-352f-4abb-af52-a597cc6457df nodeName:}" failed. No retries permitted until 2026-07-29 07:16:17.533582217 +0000 UTC m=+59.281648049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-h49x7" (UID: "80a0c65c-352f-4abb-af52-a597cc6457df") : secret "prometheus-operator-tls" not found
Jul 29 07:16:17.033862 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.033537    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq25t\" (UniqueName: \"kubernetes.io/projected/80a0c65c-352f-4abb-af52-a597cc6457df-kube-api-access-zq25t\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.034344 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.034319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80a0c65c-352f-4abb-af52-a597cc6457df-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.036031 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.036004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.043400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.043381    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq25t\" (UniqueName: \"kubernetes.io/projected/80a0c65c-352f-4abb-af52-a597cc6457df-kube-api-access-zq25t\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.537321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.537286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.539833 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.539804    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80a0c65c-352f-4abb-af52-a597cc6457df-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-h49x7\" (UID: \"80a0c65c-352f-4abb-af52-a597cc6457df\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:17.773019 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:17.772982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7"
Jul 29 07:16:18.940066 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:18.940037    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7zx5"
Jul 29 07:16:19.438253 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.438224    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-h49x7"]
Jul 29 07:16:19.441522 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:19.441489    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a0c65c_352f_4abb_af52_a597cc6457df.slice/crio-674f69860cdef21f427e9f58d71b0fc35e30cd3b7716ad3f749dfcf18dbc072d WatchSource:0}: Error finding container 674f69860cdef21f427e9f58d71b0fc35e30cd3b7716ad3f749dfcf18dbc072d: Status 404 returned error can't find the container with id 674f69860cdef21f427e9f58d71b0fc35e30cd3b7716ad3f749dfcf18dbc072d
Jul 29 07:16:19.565442 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.565409    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86b468bf49-8nrk5"]
Jul 29 07:16:19.585602 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.585534    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b468bf49-8nrk5"]
Jul 29 07:16:19.585722 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.585633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.594391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.594371    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 07:16:19.654798 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.654772    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-serving-cert\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.654913 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.654813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-oauth-config\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.654963 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.654934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-trusted-ca-bundle\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.654997 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.654961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-console-config\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.654997 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.654990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-service-ca\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.655058 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.655022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p459g\" (UniqueName: \"kubernetes.io/projected/e135beba-7621-4890-bd39-97c403ce3453-kube-api-access-p459g\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.655099 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.655072    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-oauth-serving-cert\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.755829 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.755801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-trusted-ca-bundle\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.755980 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.755836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-console-config\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.755980 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.755857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-service-ca\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.756193 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p459g\" (UniqueName: \"kubernetes.io/projected/e135beba-7621-4890-bd39-97c403ce3453-kube-api-access-p459g\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.756241 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-oauth-serving-cert\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.756306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-serving-cert\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.756361 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-oauth-config\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.756692 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-console-config\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.756955 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-service-ca\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.757043 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.756953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-trusted-ca-bundle\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.757158 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.757138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-oauth-serving-cert\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.758628 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.758608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-serving-cert\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.758745 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.758728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-oauth-config\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.764817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.764799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p459g\" (UniqueName: \"kubernetes.io/projected/e135beba-7621-4890-bd39-97c403ce3453-kube-api-access-p459g\") pod \"console-86b468bf49-8nrk5\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") " pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:19.894628 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:19.894589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:20.044468 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.044431    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6nlg" event={"ID":"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0","Type":"ContainerStarted","Data":"8a1b05df38b01cc643e108902b66f390e7492a350cfebe84372c136029a2eb24"}
Jul 29 07:16:20.045910 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.045873    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fcd6844fc-lg5m6" event={"ID":"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0","Type":"ContainerStarted","Data":"6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1"}
Jul 29 07:16:20.047176 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.047149    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7" event={"ID":"80a0c65c-352f-4abb-af52-a597cc6457df","Type":"ContainerStarted","Data":"674f69860cdef21f427e9f58d71b0fc35e30cd3b7716ad3f749dfcf18dbc072d"}
Jul 29 07:16:20.051781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.051755    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b468bf49-8nrk5"]
Jul 29 07:16:20.054738 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:20.054713    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode135beba_7621_4890_bd39_97c403ce3453.slice/crio-076eab6ddfd4f9126358368ba4309367573e2da9e5b52ee58f0294c13544e2eb WatchSource:0}: Error finding container 076eab6ddfd4f9126358368ba4309367573e2da9e5b52ee58f0294c13544e2eb: Status 404 returned error can't find the container with id 076eab6ddfd4f9126358368ba4309367573e2da9e5b52ee58f0294c13544e2eb
Jul 29 07:16:20.067429 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.067380    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fcd6844fc-lg5m6" podStartSLOduration=4.887946585 podStartE2EDuration="10.067364443s" totalImagesPullingTime="5.179417858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:20.065846052 +0000 UTC m=+61.813911894" watchObservedRunningTime="2026-07-29 07:16:20.067364443 +0000 UTC m=+61.815430262"
Jul 29 07:16:20.721397 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.721353    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:20.721575 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.721427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:20.723085 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.723034    2577 patch_prober.go:28] interesting pod/console-fcd6844fc-lg5m6 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused" start-of-body=
Jul 29 07:16:20.723206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:20.723098    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-fcd6844fc-lg5m6" podUID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" containerName="console" probeResult="failure" output="Get \"https://10.133.0.10:8443/health\": dial tcp 10.133.0.10:8443: connect: connection refused"
Jul 29 07:16:21.054090 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:21.053894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b468bf49-8nrk5" event={"ID":"e135beba-7621-4890-bd39-97c403ce3453","Type":"ContainerStarted","Data":"3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326"}
Jul 29 07:16:21.054090 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:21.053968    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b468bf49-8nrk5" event={"ID":"e135beba-7621-4890-bd39-97c403ce3453","Type":"ContainerStarted","Data":"076eab6ddfd4f9126358368ba4309367573e2da9e5b52ee58f0294c13544e2eb"}
Jul 29 07:16:21.073517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:21.073466    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86b468bf49-8nrk5" podStartSLOduration=2.073448225 podStartE2EDuration="2.073448225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:21.072389564 +0000 UTC m=+62.820455406" watchObservedRunningTime="2026-07-29 07:16:21.073448225 +0000 UTC m=+62.821514067"
Jul 29 07:16:22.059893 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:22.059849    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7" event={"ID":"80a0c65c-352f-4abb-af52-a597cc6457df","Type":"ContainerStarted","Data":"1543fc9bd83f356db67be792c97fec391727b99e40f2ad7f5fce13c9995f6344"}
Jul 29 07:16:22.060353 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:22.059932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7" event={"ID":"80a0c65c-352f-4abb-af52-a597cc6457df","Type":"ContainerStarted","Data":"9be11559d050c3b4e7f603ce450c7b3c2dec510778c1e0ac39f1eff4642058ae"}
Jul 29 07:16:22.082620 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:22.082561    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-h49x7" podStartSLOduration=4.424712561 podStartE2EDuration="6.08254026s" totalImagesPullingTime="1.657827699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:22.081333509 +0000 UTC m=+63.829399366" watchObservedRunningTime="2026-07-29 07:16:22.08254026 +0000 UTC m=+63.830606106"
Jul 29 07:16:24.068649 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.068600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-b6nlg" event={"ID":"2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0","Type":"ContainerStarted","Data":"d14daa70cd4acdc793ea9fc8bb3fb720ee8fcc803a2c56751a8fcd8c44ac29ed"}
Jul 29 07:16:24.086211 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.086164    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-b6nlg" podStartSLOduration=2.124817888 podStartE2EDuration="17.086150666s" totalImagesPullingTime="14.961332778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:16:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:24.085699212 +0000 UTC m=+65.833765055" watchObservedRunningTime="2026-07-29 07:16:24.086150666 +0000 UTC m=+65.834216510"
Jul 29 07:16:24.196038 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.196004    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"]
Jul 29 07:16:24.198852 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.198834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.202178 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.202152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-k6898\""
Jul 29 07:16:24.202306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.202152    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 07:16:24.202306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.202208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 07:16:24.213741 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.213711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"]
Jul 29 07:16:24.228390 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.228363    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xgq24"]
Jul 29 07:16:24.231543 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.231524    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.235050 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.235020    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8ph7j\""
Jul 29 07:16:24.235176 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.235036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 07:16:24.235176 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.235068    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 07:16:24.235343 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.235328    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 07:16:24.298318 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.298318 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.298527 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94mcx\" (UniqueName: \"kubernetes.io/projected/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-kube-api-access-94mcx\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.298527 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-wtmp\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298527 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.298527 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-root\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298527 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-sys\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvvhm\" (UniqueName: \"kubernetes.io/projected/6f0604e2-1137-4c01-af59-10250bdb2ed6-kube-api-access-gvvhm\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f0604e2-1137-4c01-af59-10250bdb2ed6-metrics-client-ca\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298624    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-accelerators-collector-config\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.298821 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.298680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-textfile\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.399627 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:24.399795 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.399795 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f0604e2-1137-4c01-af59-10250bdb2ed6-metrics-client-ca\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.399795 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399676    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-accelerators-collector-config\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.399795 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.399795 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:24.399780    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 07:16:24.400089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-textfile\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400089 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:24.399853    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls podName:6f0604e2-1137-4c01-af59-10250bdb2ed6 nodeName:}" failed. No retries permitted until 2026-07-29 07:16:24.899835469 +0000 UTC m=+66.647901290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls") pod "node-exporter-xgq24" (UID: "6f0604e2-1137-4c01-af59-10250bdb2ed6") : secret "node-exporter-tls" not found
Jul 29 07:16:24.400089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.400089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.399989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.400089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400018    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94mcx\" (UniqueName: \"kubernetes.io/projected/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-kube-api-access-94mcx\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.400089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-wtmp\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:24.400092    2577 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:24.400146    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-tls podName:667d0ae0-7f9c-4014-a3dd-6f5f5273d920 nodeName:}" failed. No retries permitted until 2026-07-29 07:16:24.900129614 +0000 UTC m=+66.648195454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-vrqj8" (UID: "667d0ae0-7f9c-4014-a3dd-6f5f5273d920") : secret "openshift-state-metrics-tls" not found
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-root\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-sys\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvvhm\" (UniqueName: \"kubernetes.io/projected/6f0604e2-1137-4c01-af59-10250bdb2ed6-kube-api-access-gvvhm\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f0604e2-1137-4c01-af59-10250bdb2ed6-metrics-client-ca\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-root\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-wtmp\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400394 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400400    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f0604e2-1137-4c01-af59-10250bdb2ed6-sys\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400885 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400510    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-textfile\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.400885 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-accelerators-collector-config\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.401023 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.400926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.402061 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.402036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.402248 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.402224    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.402704 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.402692    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 07:16:24.409124 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.409105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvvhm\" (UniqueName: \"kubernetes.io/projected/6f0604e2-1137-4c01-af59-10250bdb2ed6-kube-api-access-gvvhm\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.409576 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.409556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94mcx\" (UniqueName: \"kubernetes.io/projected/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-kube-api-access-94mcx\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.411797 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.411782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0270a271-eec0-4a58-a6f1-a57996c15f95-original-pull-secret\") pod \"global-pull-secret-syncer-ntlft\" (UID: \"0270a271-eec0-4a58-a6f1-a57996c15f95\") " pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:24.501249 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.501207    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:24.505042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.505022    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 07:16:24.514423 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.514405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9088f8d-af8f-497b-b4ba-4ed4ab95035b-metrics-certs\") pod \"network-metrics-daemon-bdlx9\" (UID: \"a9088f8d-af8f-497b-b4ba-4ed4ab95035b\") " pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:24.593238 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.593203    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntlft"
Jul 29 07:16:24.601679 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.601656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:24.604661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.604641    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 07:16:24.614499 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.614473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 07:16:24.624772 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.624753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsfwh\" (UniqueName: \"kubernetes.io/projected/c2ecb5e1-7181-4170-ae25-06a19a7cf1ec-kube-api-access-qsfwh\") pod \"network-check-target-b6k65\" (UID: \"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec\") " pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:24.704180 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.704154    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fb4zn\""
Jul 29 07:16:24.710778 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.710750    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-rkrh4\""
Jul 29 07:16:24.711761 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.711743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bdlx9"
Jul 29 07:16:24.716752 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.716730    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntlft"]
Jul 29 07:16:24.719034 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.719013    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:24.720508 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:24.720482    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0270a271_eec0_4a58_a6f1_a57996c15f95.slice/crio-8bfe0e1d4b57c5a3b9a3c36ea611a712dfa292ffeafe82382b83e21329275b6f WatchSource:0}: Error finding container 8bfe0e1d4b57c5a3b9a3c36ea611a712dfa292ffeafe82382b83e21329275b6f: Status 404 returned error can't find the container with id 8bfe0e1d4b57c5a3b9a3c36ea611a712dfa292ffeafe82382b83e21329275b6f
Jul 29 07:16:24.842709 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.842654    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bdlx9"]
Jul 29 07:16:24.845468 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:24.845445    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9088f8d_af8f_497b_b4ba_4ed4ab95035b.slice/crio-6d4e62142a662f1603f68a64bc9701cba3dfb7b0faf5a1f4711e2f76a1ea7ad9 WatchSource:0}: Error finding container 6d4e62142a662f1603f68a64bc9701cba3dfb7b0faf5a1f4711e2f76a1ea7ad9: Status 404 returned error can't find the container with id 6d4e62142a662f1603f68a64bc9701cba3dfb7b0faf5a1f4711e2f76a1ea7ad9
Jul 29 07:16:24.857567 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.857541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b6k65"]
Jul 29 07:16:24.860356 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:24.860326    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ecb5e1_7181_4170_ae25_06a19a7cf1ec.slice/crio-480a8e585385ec977d340dacb37cce9c4b8e9a6dab4839842cfda1a6834f5f93 WatchSource:0}: Error finding container 480a8e585385ec977d340dacb37cce9c4b8e9a6dab4839842cfda1a6834f5f93: Status 404 returned error can't find the container with id 480a8e585385ec977d340dacb37cce9c4b8e9a6dab4839842cfda1a6834f5f93
Jul 29 07:16:24.903834 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.903803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:24.903978 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.903868    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:24.904018 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:24.903984    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 07:16:24.904058 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:24.904039    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls podName:6f0604e2-1137-4c01-af59-10250bdb2ed6 nodeName:}" failed. No retries permitted until 2026-07-29 07:16:25.90402245 +0000 UTC m=+67.652088288 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls") pod "node-exporter-xgq24" (UID: "6f0604e2-1137-4c01-af59-10250bdb2ed6") : secret "node-exporter-tls" not found
Jul 29 07:16:24.906172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:24.906152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/667d0ae0-7f9c-4014-a3dd-6f5f5273d920-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-vrqj8\" (UID: \"667d0ae0-7f9c-4014-a3dd-6f5f5273d920\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:25.073069 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.072981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bdlx9" event={"ID":"a9088f8d-af8f-497b-b4ba-4ed4ab95035b","Type":"ContainerStarted","Data":"6d4e62142a662f1603f68a64bc9701cba3dfb7b0faf5a1f4711e2f76a1ea7ad9"}
Jul 29 07:16:25.076317 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.076287    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntlft" event={"ID":"0270a271-eec0-4a58-a6f1-a57996c15f95","Type":"ContainerStarted","Data":"8bfe0e1d4b57c5a3b9a3c36ea611a712dfa292ffeafe82382b83e21329275b6f"}
Jul 29 07:16:25.077664 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.077642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6k65" event={"ID":"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec","Type":"ContainerStarted","Data":"480a8e585385ec977d340dacb37cce9c4b8e9a6dab4839842cfda1a6834f5f93"}
Jul 29 07:16:25.106955 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.106918    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"
Jul 29 07:16:25.276621 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.276325    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8"]
Jul 29 07:16:25.280718 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:25.280685    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667d0ae0_7f9c_4014_a3dd_6f5f5273d920.slice/crio-e0776006978343c33e48ea35a5d1f17fff2cc242ec50b9355c6f598c070be18a WatchSource:0}: Error finding container e0776006978343c33e48ea35a5d1f17fff2cc242ec50b9355c6f598c070be18a: Status 404 returned error can't find the container with id e0776006978343c33e48ea35a5d1f17fff2cc242ec50b9355c6f598c070be18a
Jul 29 07:16:25.912758 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.912497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:25.917846 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:25.917812    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f0604e2-1137-4c01-af59-10250bdb2ed6-node-exporter-tls\") pod \"node-exporter-xgq24\" (UID: \"6f0604e2-1137-4c01-af59-10250bdb2ed6\") " pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:26.035345 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:26.035310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bdhzd"
Jul 29 07:16:26.041049 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:26.040977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xgq24"
Jul 29 07:16:26.076169 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:26.076122    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0604e2_1137_4c01_af59_10250bdb2ed6.slice/crio-7c04552eb7ec5951e115adb9ab2d9b8152ff2f0e2f1eca60b06494e4f281960d WatchSource:0}: Error finding container 7c04552eb7ec5951e115adb9ab2d9b8152ff2f0e2f1eca60b06494e4f281960d: Status 404 returned error can't find the container with id 7c04552eb7ec5951e115adb9ab2d9b8152ff2f0e2f1eca60b06494e4f281960d
Jul 29 07:16:26.086135 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:26.086058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgq24" event={"ID":"6f0604e2-1137-4c01-af59-10250bdb2ed6","Type":"ContainerStarted","Data":"7c04552eb7ec5951e115adb9ab2d9b8152ff2f0e2f1eca60b06494e4f281960d"}
Jul 29 07:16:26.088704 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:26.088674    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8" event={"ID":"667d0ae0-7f9c-4014-a3dd-6f5f5273d920","Type":"ContainerStarted","Data":"29dcc6597e5d726271dedb18d60d47c5364842048275fe216f3a11890497b5cb"}
Jul 29 07:16:26.088819 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:26.088740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8" event={"ID":"667d0ae0-7f9c-4014-a3dd-6f5f5273d920","Type":"ContainerStarted","Data":"cecbd972a32acd9e5612e45872275a3865bf470d515df14dfcaf2e7fc0e193ee"}
Jul 29 07:16:26.088819 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:26.088771    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8" event={"ID":"667d0ae0-7f9c-4014-a3dd-6f5f5273d920","Type":"ContainerStarted","Data":"e0776006978343c33e48ea35a5d1f17fff2cc242ec50b9355c6f598c070be18a"}
Jul 29 07:16:27.103130 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:27.103032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bdlx9" event={"ID":"a9088f8d-af8f-497b-b4ba-4ed4ab95035b","Type":"ContainerStarted","Data":"95659db5b79ba61b3c35beea689514186f52d9832051b3542241d28fa864dec9"}
Jul 29 07:16:27.103130 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:27.103108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bdlx9" event={"ID":"a9088f8d-af8f-497b-b4ba-4ed4ab95035b","Type":"ContainerStarted","Data":"e6178b585c13a1cb67d0ca4e0b21934d813e71fb19c634e3998c76db3aa44b1d"}
Jul 29 07:16:27.122106 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:27.122049    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bdlx9" podStartSLOduration=67.535709131 podStartE2EDuration="1m9.122028456s" totalImagesPullingTime="1.586319325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:27.1206503 +0000 UTC m=+68.868716146" watchObservedRunningTime="2026-07-29 07:16:27.122028456 +0000 UTC m=+68.870094301"
Jul 29 07:16:28.638054 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.638016    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c788db9b-hr9sb"]
Jul 29 07:16:28.640259 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.640239    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.643154 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.643126    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 29 07:16:28.644377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.644352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 29 07:16:28.644531 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.644351    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 29 07:16:28.644726 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.644710    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 29 07:16:28.644801 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.644788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ktg4m\""
Jul 29 07:16:28.644856 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.644802    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7k2r6ii6m0p4o\""
Jul 29 07:16:28.652976 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.652953    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c788db9b-hr9sb"]
Jul 29 07:16:28.739795 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.739754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.739995 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.739820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-client-ca-bundle\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.739995 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.739854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-secret-metrics-server-tls\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.739995 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.739882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-metrics-server-audit-profiles\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.739995 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.739929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4ngx\" (UniqueName: \"kubernetes.io/projected/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-kube-api-access-j4ngx\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.740179 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.740001    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-secret-metrics-server-client-certs\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.740179 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.740075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-audit-log\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841217 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-audit-log\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841269    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-client-ca-bundle\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-secret-metrics-server-tls\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841324    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-metrics-server-audit-profiles\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841391 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4ngx\" (UniqueName: \"kubernetes.io/projected/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-kube-api-access-j4ngx\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-secret-metrics-server-client-certs\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.841661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.841634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-audit-log\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.842086 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.842050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.842549 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.842517    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-metrics-server-audit-profiles\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.844182 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.844157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-secret-metrics-server-client-certs\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.844399 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.844372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-secret-metrics-server-tls\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.844474 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.844379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-client-ca-bundle\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.854625 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.854599    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4ngx\" (UniqueName: \"kubernetes.io/projected/78fd41e8-490a-4e1a-a3ae-1006fb0a6055-kube-api-access-j4ngx\") pod \"metrics-server-5c788db9b-hr9sb\" (UID: \"78fd41e8-490a-4e1a-a3ae-1006fb0a6055\") " pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:28.951126 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:28.950549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:29.011431 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.011383    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"]
Jul 29 07:16:29.013399 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.013378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.016297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.016269    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 07:16:29.016599 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.016579    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-qp7jn\""
Jul 29 07:16:29.028430 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.028400    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"]
Jul 29 07:16:29.144967 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.144927    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a44532ec-4b0e-4b23-bd5f-05dd0c34157d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-pcbf6\" (UID: \"a44532ec-4b0e-4b23-bd5f-05dd0c34157d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.145142 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.145009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wblp\" (UniqueName: \"kubernetes.io/projected/a44532ec-4b0e-4b23-bd5f-05dd0c34157d-kube-api-access-6wblp\") pod \"monitoring-plugin-68b7fc9574-pcbf6\" (UID: \"a44532ec-4b0e-4b23-bd5f-05dd0c34157d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.246061 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.245967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a44532ec-4b0e-4b23-bd5f-05dd0c34157d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-pcbf6\" (UID: \"a44532ec-4b0e-4b23-bd5f-05dd0c34157d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.246061 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.246060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wblp\" (UniqueName: \"kubernetes.io/projected/a44532ec-4b0e-4b23-bd5f-05dd0c34157d-kube-api-access-6wblp\") pod \"monitoring-plugin-68b7fc9574-pcbf6\" (UID: \"a44532ec-4b0e-4b23-bd5f-05dd0c34157d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.248811 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.248782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a44532ec-4b0e-4b23-bd5f-05dd0c34157d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-pcbf6\" (UID: \"a44532ec-4b0e-4b23-bd5f-05dd0c34157d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.255789 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.255763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wblp\" (UniqueName: \"kubernetes.io/projected/a44532ec-4b0e-4b23-bd5f-05dd0c34157d-kube-api-access-6wblp\") pod \"monitoring-plugin-68b7fc9574-pcbf6\" (UID: \"a44532ec-4b0e-4b23-bd5f-05dd0c34157d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.325747 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.325710    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:29.894687 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.894661    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:29.895052 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.894721    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:29.899428 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:29.899406    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:30.095164 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.095075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"]
Jul 29 07:16:30.098943 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:30.098875    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda44532ec_4b0e_4b23_bd5f_05dd0c34157d.slice/crio-7a40602d42581cb7df8f4d9b5d0390333713d5c5b994b6d99609ab70c417f8dc WatchSource:0}: Error finding container 7a40602d42581cb7df8f4d9b5d0390333713d5c5b994b6d99609ab70c417f8dc: Status 404 returned error can't find the container with id 7a40602d42581cb7df8f4d9b5d0390333713d5c5b994b6d99609ab70c417f8dc
Jul 29 07:16:30.110830 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.110798    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c788db9b-hr9sb"]
Jul 29 07:16:30.113808 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:30.113766    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fd41e8_490a_4e1a_a3ae_1006fb0a6055.slice/crio-1df386e3ef46e4c69f4660564398b886672b1f0bf67bb8b3affa47fd3a75b321 WatchSource:0}: Error finding container 1df386e3ef46e4c69f4660564398b886672b1f0bf67bb8b3affa47fd3a75b321: Status 404 returned error can't find the container with id 1df386e3ef46e4c69f4660564398b886672b1f0bf67bb8b3affa47fd3a75b321
Jul 29 07:16:30.118836 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.118519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgq24" event={"ID":"6f0604e2-1137-4c01-af59-10250bdb2ed6","Type":"ContainerStarted","Data":"5486ed2a15ceae8835cd28546690452dd3e96bf6e3ebcbea0a4dde638e1b29c1"}
Jul 29 07:16:30.120806 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.120726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntlft" event={"ID":"0270a271-eec0-4a58-a6f1-a57996c15f95","Type":"ContainerStarted","Data":"af2d779d2d0581c9346f32b400a9824b8871813a9feb14f29d6a61c3c47a9938"}
Jul 29 07:16:30.123717 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.123695    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8" event={"ID":"667d0ae0-7f9c-4014-a3dd-6f5f5273d920","Type":"ContainerStarted","Data":"98bb1407dc2dfa2d19000250a6347db4a2fe1bc889554757c008a4da8b36e67c"}
Jul 29 07:16:30.124881 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.124858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6" event={"ID":"a44532ec-4b0e-4b23-bd5f-05dd0c34157d","Type":"ContainerStarted","Data":"7a40602d42581cb7df8f4d9b5d0390333713d5c5b994b6d99609ab70c417f8dc"}
Jul 29 07:16:30.126656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.126625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b6k65" event={"ID":"c2ecb5e1-7181-4170-ae25-06a19a7cf1ec","Type":"ContainerStarted","Data":"17ce33dcc7749dfa216ed3147319a31bcc31bf15945371fe8433ef508a7d0593"}
Jul 29 07:16:30.126803 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.126788    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:16:30.131742 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.131711    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:16:30.173625 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.173565    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ntlft" podStartSLOduration=66.96424136 podStartE2EDuration="1m12.173543528s" totalImagesPullingTime="5.209302168s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:30.172408521 +0000 UTC m=+71.920474380" watchObservedRunningTime="2026-07-29 07:16:30.173543528 +0000 UTC m=+71.921609371"
Jul 29 07:16:30.190057 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.190029    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fcd6844fc-lg5m6"]
Jul 29 07:16:30.202616 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.202571    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-vrqj8" podStartSLOduration=1.712542557 podStartE2EDuration="6.202550612s" totalImagesPullingTime="4.490008055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:30.201859381 +0000 UTC m=+71.949925223" watchObservedRunningTime="2026-07-29 07:16:30.202550612 +0000 UTC m=+71.950616453"
Jul 29 07:16:30.222269 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.222224    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b6k65" podStartSLOduration=67.156694462 podStartE2EDuration="1m12.222208669s" totalImagesPullingTime="5.065514207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:30.221585614 +0000 UTC m=+71.969651458" watchObservedRunningTime="2026-07-29 07:16:30.222208669 +0000 UTC m=+71.970274511"
Jul 29 07:16:30.463443 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.463355    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:16:30.467507 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.467482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.471284 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.471259    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 07:16:30.471591 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.471569    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 07:16:30.471804 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.471782    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 07:16:30.472195 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.471783    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 07:16:30.472666 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.471952    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 07:16:30.472948 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.472443    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 07:16:30.473476 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.472494    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 07:16:30.473726 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.473706    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 07:16:30.473836 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.473816    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 07:16:30.474142 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.474123    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5vugoo425avs0\""
Jul 29 07:16:30.474231 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.474161    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 07:16:30.474472 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.474456    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 07:16:30.474559 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.474511    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 07:16:30.474717 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.474699    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s8tfc\""
Jul 29 07:16:30.476669 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.476645    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 07:16:30.479647 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.479624    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 07:16:30.488601 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.488580    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:16:30.559268 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559358    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559487    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559543    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.559784 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559769    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knbdn\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-kube-api-access-knbdn\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.560269 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.560269 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.560269 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.559854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661218    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knbdn\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-kube-api-access-knbdn\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661282    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661423    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.661513 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662021 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.661760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662869 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.662680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.662869 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.662795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.663628 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.663575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.664608 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.664582    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.667570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.665147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.667570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.666883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.667570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.667075    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.667570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.667265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.667570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.667509    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.667570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.667540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.668188 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.667649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.668813 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.668722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.670540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.670569    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.671053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.671200    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-web-config\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.671236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config-out\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671590 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.671570    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.671968 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.671940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.672747 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.672709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knbdn\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-kube-api-access-knbdn\") pod \"prometheus-k8s-0\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.778658 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.778573    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:30.950375 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:30.950092    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:16:30.954722 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:16:30.954686    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c86aa9_1c07_4ef0_b0c4_72f9ae077aa5.slice/crio-4fe91c4cd39b83409a789e3b3cbb65568cf4e189e9507d097ac6fe31d5dbc4ea WatchSource:0}: Error finding container 4fe91c4cd39b83409a789e3b3cbb65568cf4e189e9507d097ac6fe31d5dbc4ea: Status 404 returned error can't find the container with id 4fe91c4cd39b83409a789e3b3cbb65568cf4e189e9507d097ac6fe31d5dbc4ea
Jul 29 07:16:31.132480 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:31.132293    2577 generic.go:356] "Generic (PLEG): container finished" podID="6f0604e2-1137-4c01-af59-10250bdb2ed6" containerID="5486ed2a15ceae8835cd28546690452dd3e96bf6e3ebcbea0a4dde638e1b29c1" exitCode=0
Jul 29 07:16:31.132480 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:31.132338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgq24" event={"ID":"6f0604e2-1137-4c01-af59-10250bdb2ed6","Type":"ContainerDied","Data":"5486ed2a15ceae8835cd28546690452dd3e96bf6e3ebcbea0a4dde638e1b29c1"}
Jul 29 07:16:31.134300 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:31.134270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb" event={"ID":"78fd41e8-490a-4e1a-a3ae-1006fb0a6055","Type":"ContainerStarted","Data":"1df386e3ef46e4c69f4660564398b886672b1f0bf67bb8b3affa47fd3a75b321"}
Jul 29 07:16:31.135855 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:31.135787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"4fe91c4cd39b83409a789e3b3cbb65568cf4e189e9507d097ac6fe31d5dbc4ea"}
Jul 29 07:16:32.143201 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.143158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgq24" event={"ID":"6f0604e2-1137-4c01-af59-10250bdb2ed6","Type":"ContainerStarted","Data":"f5aec520b1f767140498efab95bf6126eebbf3f09d4caff73bcc18d27b8c213e"}
Jul 29 07:16:32.143634 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.143263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xgq24" event={"ID":"6f0604e2-1137-4c01-af59-10250bdb2ed6","Type":"ContainerStarted","Data":"62618d52cb80c449439b6a79d131662b5f32f5e36ff854b29becf02492ac281e"}
Jul 29 07:16:32.146226 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.145700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6" event={"ID":"a44532ec-4b0e-4b23-bd5f-05dd0c34157d","Type":"ContainerStarted","Data":"521a9c8ba547ec450871e56b285f13c4e522cabb6b8d4df0b9b144f5691ddabd"}
Jul 29 07:16:32.146226 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.146014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:32.147878 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.147842    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb" event={"ID":"78fd41e8-490a-4e1a-a3ae-1006fb0a6055","Type":"ContainerStarted","Data":"ef17a177f4f10a6511621073c1768fe8a25957ff42d1c55fe81ff329495ff19b"}
Jul 29 07:16:32.151684 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.151665    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6"
Jul 29 07:16:32.193557 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.193495    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xgq24" podStartSLOduration=4.260440382 podStartE2EDuration="8.193472729s" totalImagesPullingTime="3.846599552s" totalInitContainerRuntime="86.432795ms" isStatefulPod=true podCreationTimestamp="2026-07-29 07:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:32.190782209 +0000 UTC m=+73.938848077" watchObservedRunningTime="2026-07-29 07:16:32.193472729 +0000 UTC m=+73.941538572"
Jul 29 07:16:32.273159 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.273109    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb" podStartSLOduration=2.520946764 podStartE2EDuration="4.273086759s" totalImagesPullingTime="1.752139995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:32.272745317 +0000 UTC m=+74.020811159" watchObservedRunningTime="2026-07-29 07:16:32.273086759 +0000 UTC m=+74.021152602"
Jul 29 07:16:32.319361 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:32.319309    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-pcbf6" podStartSLOduration=2.555350571 podStartE2EDuration="4.319294104s" totalImagesPullingTime="1.763943533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:32.318129782 +0000 UTC m=+74.066195625" watchObservedRunningTime="2026-07-29 07:16:32.319294104 +0000 UTC m=+74.067359946"
Jul 29 07:16:33.153056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:33.153024    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1" exitCode=0
Jul 29 07:16:33.153532 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:33.153120    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"}
Jul 29 07:16:37.178946 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:37.177998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"}
Jul 29 07:16:37.178946 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:37.178116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"}
Jul 29 07:16:39.189746 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:39.189713    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"}
Jul 29 07:16:39.190138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:39.189782    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"}
Jul 29 07:16:39.190138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:39.189821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"}
Jul 29 07:16:39.190138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:39.189864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerStarted","Data":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"}
Jul 29 07:16:39.230004 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:39.229946    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.239022721 podStartE2EDuration="9.229928809s" totalImagesPullingTime="6.969367785s" totalInitContainerRuntime="21.538303ms" isStatefulPod=false podCreationTimestamp="2026-07-29 07:16:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:16:39.227339801 +0000 UTC m=+80.975405643" watchObservedRunningTime="2026-07-29 07:16:39.229928809 +0000 UTC m=+80.977994651"
Jul 29 07:16:40.779321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:40.779193    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:16:46.504892 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:46.504853    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b468bf49-8nrk5"]
Jul 29 07:16:48.951611 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:48.951575    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:48.952091 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:48.951644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:16:55.211548 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.211471    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fcd6844fc-lg5m6" podUID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" containerName="console" containerID="cri-o://6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1" gracePeriod=15
Jul 29 07:16:55.449925 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.449885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:55.465696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.465637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-oauth-config\") pod \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") "
Jul 29 07:16:55.465696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.465676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-oauth-serving-cert\") pod \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") "
Jul 29 07:16:55.465845 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.465704    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-serving-cert\") pod \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") "
Jul 29 07:16:55.465845 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.465748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9trj\" (UniqueName: \"kubernetes.io/projected/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-kube-api-access-g9trj\") pod \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") "
Jul 29 07:16:55.465947 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.465888    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-config\") pod \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") "
Jul 29 07:16:55.466006 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.465967    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-service-ca\") pod \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\" (UID: \"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0\") "
Jul 29 07:16:55.466213 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.466170    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" (UID: "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:16:55.466301 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.466234    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-config" (OuterVolumeSpecName: "console-config") pod "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" (UID: "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:16:55.466881 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.466527    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" (UID: "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:16:55.468617 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.468589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" (UID: "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:16:55.468744 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.468702    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-kube-api-access-g9trj" (OuterVolumeSpecName: "kube-api-access-g9trj") pod "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" (UID: "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0"). InnerVolumeSpecName "kube-api-access-g9trj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:16:55.468842 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.468816    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" (UID: "1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:16:55.566961 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.566896    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9trj\" (UniqueName: \"kubernetes.io/projected/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-kube-api-access-g9trj\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:16:55.566961 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.566954    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-config\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:16:55.566961 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.566967    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:16:55.567205 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.566977    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-oauth-config\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:16:55.567205 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.566989    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-oauth-serving-cert\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:16:55.567205 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:55.567001    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0-console-serving-cert\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:16:56.255886 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.255848    2577 generic.go:356] "Generic (PLEG): container finished" podID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" containerID="6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1" exitCode=0
Jul 29 07:16:56.256339 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.255915    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fcd6844fc-lg5m6"
Jul 29 07:16:56.256339 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.255937    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fcd6844fc-lg5m6" event={"ID":"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0","Type":"ContainerDied","Data":"6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1"}
Jul 29 07:16:56.256339 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.255996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fcd6844fc-lg5m6" event={"ID":"1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0","Type":"ContainerDied","Data":"ea9e07b8278f3b98f637a8357502fbe5f87cec7f9c6c0094088dcef74dc39644"}
Jul 29 07:16:56.256339 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.256032    2577 scope.go:117] "RemoveContainer" containerID="6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1"
Jul 29 07:16:56.264649 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.264627    2577 scope.go:117] "RemoveContainer" containerID="6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1"
Jul 29 07:16:56.264931 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:16:56.264891    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1\": container with ID starting with 6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1 not found: ID does not exist" containerID="6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1"
Jul 29 07:16:56.264992 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.264940    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1"} err="failed to get container status \"6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1\": rpc error: code = NotFound desc = could not find container \"6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1\": container with ID starting with 6427b41cfda261fe27cdd8804186982f51c8d0b81d82cc7e77863ebf451686e1 not found: ID does not exist"
Jul 29 07:16:56.277465 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.277439    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fcd6844fc-lg5m6"]
Jul 29 07:16:56.281577 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.281554    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fcd6844fc-lg5m6"]
Jul 29 07:16:56.788766 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:16:56.788733    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" path="/var/lib/kubelet/pods/1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0/volumes"
Jul 29 07:17:01.138842 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:01.138812    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b6k65"
Jul 29 07:17:08.957273 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:08.957234    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:17:08.961290 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:08.961268    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c788db9b-hr9sb"
Jul 29 07:17:11.529420 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.529353    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86b468bf49-8nrk5" podUID="e135beba-7621-4890-bd39-97c403ce3453" containerName="console" containerID="cri-o://3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326" gracePeriod=15
Jul 29 07:17:11.800071 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.800014    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:17:11.887547 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-console-config\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887555    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-oauth-config\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887592    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p459g\" (UniqueName: \"kubernetes.io/projected/e135beba-7621-4890-bd39-97c403ce3453-kube-api-access-p459g\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887636    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-serving-cert\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887669    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-trusted-ca-bundle\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887954 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-oauth-serving-cert\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887954 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-service-ca\") pod \"e135beba-7621-4890-bd39-97c403ce3453\" (UID: \"e135beba-7621-4890-bd39-97c403ce3453\") "
Jul 29 07:17:11.887954 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.887932    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-console-config" (OuterVolumeSpecName: "console-config") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:11.888174 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.888154    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-console-config\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:11.888237 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.888205    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:11.888237 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.888223    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:11.888339 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.888262    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-service-ca" (OuterVolumeSpecName: "service-ca") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:11.889808 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.889782    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:11.889971 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.889863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:11.889971 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.889888    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e135beba-7621-4890-bd39-97c403ce3453-kube-api-access-p459g" (OuterVolumeSpecName: "kube-api-access-p459g") pod "e135beba-7621-4890-bd39-97c403ce3453" (UID: "e135beba-7621-4890-bd39-97c403ce3453"). InnerVolumeSpecName "kube-api-access-p459g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:17:11.989400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.989369    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p459g\" (UniqueName: \"kubernetes.io/projected/e135beba-7621-4890-bd39-97c403ce3453-kube-api-access-p459g\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:11.989400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.989394    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-serving-cert\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:11.989400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.989403    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-trusted-ca-bundle\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:11.989624 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.989413    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-oauth-serving-cert\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:11.989624 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.989423    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e135beba-7621-4890-bd39-97c403ce3453-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:11.989624 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:11.989432    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e135beba-7621-4890-bd39-97c403ce3453-console-oauth-config\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:12.317876 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.317840    2577 generic.go:356] "Generic (PLEG): container finished" podID="e135beba-7621-4890-bd39-97c403ce3453" containerID="3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326" exitCode=0
Jul 29 07:17:12.318138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.317886    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b468bf49-8nrk5"
Jul 29 07:17:12.318138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.317926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b468bf49-8nrk5" event={"ID":"e135beba-7621-4890-bd39-97c403ce3453","Type":"ContainerDied","Data":"3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326"}
Jul 29 07:17:12.318138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.317976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b468bf49-8nrk5" event={"ID":"e135beba-7621-4890-bd39-97c403ce3453","Type":"ContainerDied","Data":"076eab6ddfd4f9126358368ba4309367573e2da9e5b52ee58f0294c13544e2eb"}
Jul 29 07:17:12.318138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.317996    2577 scope.go:117] "RemoveContainer" containerID="3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326"
Jul 29 07:17:12.326689 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.326668    2577 scope.go:117] "RemoveContainer" containerID="3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326"
Jul 29 07:17:12.326959 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:12.326939    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326\": container with ID starting with 3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326 not found: ID does not exist" containerID="3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326"
Jul 29 07:17:12.327032 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.326968    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326"} err="failed to get container status \"3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326\": rpc error: code = NotFound desc = could not find container \"3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326\": container with ID starting with 3f7f6c7b4188866af1f55d37fdcfa8a60dfd5df1834fc8466bda34b8311c3326 not found: ID does not exist"
Jul 29 07:17:12.338864 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.338843    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b468bf49-8nrk5"]
Jul 29 07:17:12.343585 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.343565    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86b468bf49-8nrk5"]
Jul 29 07:17:12.789591 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:12.789556    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e135beba-7621-4890-bd39-97c403ce3453" path="/var/lib/kubelet/pods/e135beba-7621-4890-bd39-97c403ce3453/volumes"
Jul 29 07:17:30.779048 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:30.779003    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:30.798774 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:30.798743    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:31.400477 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:31.400450    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:48.916310 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916270    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:17:48.916831 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916660    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy" containerID="cri-o://d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd" gracePeriod=600
Jul 29 07:17:48.916831 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916665    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="thanos-sidecar" containerID="cri-o://6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57" gracePeriod=600
Jul 29 07:17:48.916831 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916697    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-web" containerID="cri-o://7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9" gracePeriod=600
Jul 29 07:17:48.916831 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916717    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60" gracePeriod=600
Jul 29 07:17:48.916831 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916724    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="config-reloader" containerID="cri-o://e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e" gracePeriod=600
Jul 29 07:17:48.917113 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:48.916660    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="prometheus" containerID="cri-o://6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10" gracePeriod=600
Jul 29 07:17:49.185227 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.185197    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.290202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290167    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-serving-certs-ca-bundle\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290212    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-kubelet-serving-ca-bundle\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-1\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290260    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-tls\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290294    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290323    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knbdn\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-kube-api-access-knbdn\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.290705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290630    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.290705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290642    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.290705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290692    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-grpc-tls\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290921 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290740    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-thanos-prometheus-http-client-file\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290921 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-web-config\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290921 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290818    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-kube-rbac-proxy\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290921 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290842    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.290921 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290877    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-metrics-client-ca\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290928    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-metrics-client-certs\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.290973    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-2\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291003    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-tls-assets\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291027    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-trusted-ca-bundle\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291064    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-0\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config-out\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291149    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-db\") pod \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\" (UID: \"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5\") "
Jul 29 07:17:49.291530 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291391    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.291530 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291410    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.291530 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.291426    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.292662 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.292623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.293016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.292987    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.293463 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.293432    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:17:49.293552 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.293509    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.293619 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.293587    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-kube-api-access-knbdn" (OuterVolumeSpecName: "kube-api-access-knbdn") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "kube-api-access-knbdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:17:49.294007 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.293982    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.294144 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.294123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 07:17:49.294241 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.294145    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config" (OuterVolumeSpecName: "config") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.294312 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.294280    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.294312 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.294248    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.294437 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.294412    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.295158 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.295135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.295541 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.295518    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.295789 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.295763    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:17:49.295789 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.295785    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config-out" (OuterVolumeSpecName: "config-out") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:17:49.296089 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.296072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.304381 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.304350    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-web-config" (OuterVolumeSpecName: "web-config") pod "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" (UID: "43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:17:49.392005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.391965    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-web-config\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392000    2577 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-kube-rbac-proxy\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392012    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392022    2577 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-configmap-metrics-client-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392030    2577 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-metrics-client-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392038    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392047    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-tls-assets\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392055    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392063    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392071    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config-out\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392079    2577 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-prometheus-k8s-db\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392087    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-tls\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392094    2577 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-config\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392102    2577 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392111    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knbdn\" (UniqueName: \"kubernetes.io/projected/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-kube-api-access-knbdn\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392118    2577 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-secret-grpc-tls\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.392252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.392127    2577 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5-thanos-prometheus-http-client-file\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452441    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60" exitCode=0
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452466    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd" exitCode=0
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452472    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9" exitCode=0
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452479    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57" exitCode=0
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452484    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e" exitCode=0
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452490    2577 generic.go:356] "Generic (PLEG): container finished" podID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10" exitCode=0
Jul 29 07:17:49.452521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452512    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452517    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"}
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"}
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452653    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"}
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"}
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452692    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"}
Jul 29 07:17:49.452817 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"}
Jul 29 07:17:49.453147 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.452833    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5","Type":"ContainerDied","Data":"4fe91c4cd39b83409a789e3b3cbb65568cf4e189e9507d097ac6fe31d5dbc4ea"}
Jul 29 07:17:49.461250 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.461052    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.468423 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.468404    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.475604 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.475581    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.477316 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.477295    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:17:49.481604 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.481583    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:17:49.482494 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.482478    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.488732 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.488715    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.495375 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.495359    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.501717 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.501696    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.501982 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.501962    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.502062 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.501994    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"} err="failed to get container status \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist"
Jul 29 07:17:49.502062 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502020    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.502272 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.502255    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.502315 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502278    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"} err="failed to get container status \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist"
Jul 29 07:17:49.502315 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502293    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.502508 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.502491    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.502570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502517    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"} err="failed to get container status \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist"
Jul 29 07:17:49.502570 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502538    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.502776 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.502760    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.502815 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502783    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"} err="failed to get container status \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist"
Jul 29 07:17:49.502815 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.502797    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.503089 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.503068    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.503149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503093    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"} err="failed to get container status \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist"
Jul 29 07:17:49.503149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503105    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.503327 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.503309    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.503383 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503328    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"} err="failed to get container status \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist"
Jul 29 07:17:49.503383 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503339    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.503536 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:17:49.503519    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.503600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503543    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"} err="failed to get container status \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist"
Jul 29 07:17:49.503600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503563    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.503788 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503772    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"} err="failed to get container status \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist"
Jul 29 07:17:49.503830 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503790    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.504001 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.503987    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"} err="failed to get container status \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist"
Jul 29 07:17:49.504001 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504001    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.504218 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504199    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"} err="failed to get container status \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist"
Jul 29 07:17:49.504288 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504220    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.504398 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504383    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"} err="failed to get container status \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist"
Jul 29 07:17:49.504441 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504398    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.504582 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504563    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"} err="failed to get container status \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist"
Jul 29 07:17:49.504627 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504583    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.504807 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504787    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"} err="failed to get container status \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist"
Jul 29 07:17:49.504849 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.504809    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.505029 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505010    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"} err="failed to get container status \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist"
Jul 29 07:17:49.505101 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505030    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.505270 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505252    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"} err="failed to get container status \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist"
Jul 29 07:17:49.505319 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505270    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.505461 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505444    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"} err="failed to get container status \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist"
Jul 29 07:17:49.505505 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505462    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.505653 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505638    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"} err="failed to get container status \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist"
Jul 29 07:17:49.505695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505653    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.505828 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505808    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"} err="failed to get container status \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist"
Jul 29 07:17:49.505908 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.505828    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.506070 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506055    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"} err="failed to get container status \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist"
Jul 29 07:17:49.506070 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506070    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.506233 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506215    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"} err="failed to get container status \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist"
Jul 29 07:17:49.506295 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506234    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.506469 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506450    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"} err="failed to get container status \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist"
Jul 29 07:17:49.506509 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506470    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.506755 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506727    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"} err="failed to get container status \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist"
Jul 29 07:17:49.506847 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.506756    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.507149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507123    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"} err="failed to get container status \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist"
Jul 29 07:17:49.507228 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507151    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.507437 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507416    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"} err="failed to get container status \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist"
Jul 29 07:17:49.507437 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507437    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.507700 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507677    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"} err="failed to get container status \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist"
Jul 29 07:17:49.507752 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507703    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.507993 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507977    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"} err="failed to get container status \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist"
Jul 29 07:17:49.507993 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.507993    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.508219 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508202    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"} err="failed to get container status \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist"
Jul 29 07:17:49.508262 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508220    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.508476 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508452    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"} err="failed to get container status \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist"
Jul 29 07:17:49.508519 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508476    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.508685 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508670    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:17:49.508725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508683    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"} err="failed to get container status \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist"
Jul 29 07:17:49.508725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508701    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.508892 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508876    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"} err="failed to get container status \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist"
Jul 29 07:17:49.508965 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508892    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.508965 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508955    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="init-config-reloader"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508967    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="init-config-reloader"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508976    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="thanos-sidecar"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508982    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="thanos-sidecar"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508989    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="config-reloader"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.508994    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="config-reloader"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509006    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" containerName="console"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509014    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" containerName="console"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509025    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-thanos"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509033    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-thanos"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509041    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="prometheus"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509046    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="prometheus"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509051    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509056    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy"
Jul 29 07:17:49.509056 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509064    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e135beba-7621-4890-bd39-97c403ce3453" containerName="console"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509071    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e135beba-7621-4890-bd39-97c403ce3453" containerName="console"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509083    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-web"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509088    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-web"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509133    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="config-reloader"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509140    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-thanos"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509149    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e135beba-7621-4890-bd39-97c403ce3453" containerName="console"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509152    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"} err="failed to get container status \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509170    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509159    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="thanos-sidecar"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509220    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy-web"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509230    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="kube-rbac-proxy"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509237    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f0a7571-4fa1-44c0-a0d0-6616b5a7cdf0" containerName="console"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509244    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" containerName="prometheus"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509434    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"} err="failed to get container status \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist"
Jul 29 07:17:49.509517 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509452    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.510036 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509626    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"} err="failed to get container status \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist"
Jul 29 07:17:49.510036 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509640    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.510036 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509828    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"} err="failed to get container status \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist"
Jul 29 07:17:49.510036 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.509851    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.510213 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.510160    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"} err="failed to get container status \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist"
Jul 29 07:17:49.510213 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.510184    2577 scope.go:117] "RemoveContainer" containerID="275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"
Jul 29 07:17:49.510572 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.510551    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60"} err="failed to get container status \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": rpc error: code = NotFound desc = could not find container \"275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60\": container with ID starting with 275efbe96af12f714cc22a3285dbaa0afaccdeff1fca215f577cef976d215f60 not found: ID does not exist"
Jul 29 07:17:49.510572 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.510573    2577 scope.go:117] "RemoveContainer" containerID="d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"
Jul 29 07:17:49.510848 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.510827    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd"} err="failed to get container status \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": rpc error: code = NotFound desc = could not find container \"d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd\": container with ID starting with d46eb2789329f979f38e284da58171c5ffd9d43fe9f298dfe0656fe3134261bd not found: ID does not exist"
Jul 29 07:17:49.510848 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.510847    2577 scope.go:117] "RemoveContainer" containerID="7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"
Jul 29 07:17:49.511171 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511147    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9"} err="failed to get container status \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": rpc error: code = NotFound desc = could not find container \"7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9\": container with ID starting with 7dc579342f6c875ceb80a3f56240066ef9af81d298f0eb46d30d346203d4d6b9 not found: ID does not exist"
Jul 29 07:17:49.511237 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511173    2577 scope.go:117] "RemoveContainer" containerID="6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"
Jul 29 07:17:49.511409 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511387    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57"} err="failed to get container status \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": rpc error: code = NotFound desc = could not find container \"6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57\": container with ID starting with 6986460a93c58ad45fccb4634a4529bb2d7dfb9eebb19d61752d99a8d299bd57 not found: ID does not exist"
Jul 29 07:17:49.511472 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511412    2577 scope.go:117] "RemoveContainer" containerID="e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"
Jul 29 07:17:49.511629 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511612    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e"} err="failed to get container status \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": rpc error: code = NotFound desc = could not find container \"e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e\": container with ID starting with e556f54967519044abd92c4945da3ad9374296f93a9e086e6b07b5332ccf860e not found: ID does not exist"
Jul 29 07:17:49.511675 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511630    2577 scope.go:117] "RemoveContainer" containerID="6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"
Jul 29 07:17:49.511819 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511804    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10"} err="failed to get container status \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": rpc error: code = NotFound desc = could not find container \"6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10\": container with ID starting with 6ea10f0de2c3ff92202976ae1165cc143b1e15a95ecd945e1204f4b38537de10 not found: ID does not exist"
Jul 29 07:17:49.511860 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.511820    2577 scope.go:117] "RemoveContainer" containerID="683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"
Jul 29 07:17:49.512031 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.512015    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1"} err="failed to get container status \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": rpc error: code = NotFound desc = could not find container \"683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1\": container with ID starting with 683ebdd572b665162024e5785ab47a0df478aada4863720b9d8863e2cfa8d0e1 not found: ID does not exist"
Jul 29 07:17:49.512986 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.512972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.516086 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.515678    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 29 07:17:49.516086 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.515919    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 29 07:17:49.516220 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.516139    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 29 07:17:49.516280 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.516262    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 29 07:17:49.516381 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.516362    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 29 07:17:49.516515 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.516390    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 29 07:17:49.516515 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.516451    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 29 07:17:49.516515 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.516480    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 29 07:17:49.517781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.517633    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-s8tfc\""
Jul 29 07:17:49.517781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.517690    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 29 07:17:49.517781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.517723    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 29 07:17:49.517781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.517762    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 29 07:17:49.518167 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.517854    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 29 07:17:49.518167 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.518050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5vugoo425avs0\""
Jul 29 07:17:49.519273 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.519252    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 29 07:17:49.523179 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.523161    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 29 07:17:49.531603 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.531579    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:17:49.593751 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.593751 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-web-config\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593863    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fdc27892-e7ef-41a2-b3ea-4a96d0753927-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593937    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594042 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.593993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594047    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-config\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fdc27892-e7ef-41a2-b3ea-4a96d0753927-config-out\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594315    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lffjg\" (UniqueName: \"kubernetes.io/projected/fdc27892-e7ef-41a2-b3ea-4a96d0753927-kube-api-access-lffjg\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.594521 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.594360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.694989 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.694938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.694989 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.694994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fdc27892-e7ef-41a2-b3ea-4a96d0753927-config-out\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lffjg\" (UniqueName: \"kubernetes.io/projected/fdc27892-e7ef-41a2-b3ea-4a96d0753927-kube-api-access-lffjg\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695069    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695131    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695206 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-web-config\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695245    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fdc27892-e7ef-41a2-b3ea-4a96d0753927-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695352    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695429    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.695536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-config\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.696095 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.695975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.696208 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.696185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.696774 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.696747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.697447 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.697419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.697938 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.697914    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698216 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698122    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698216 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698366 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698264    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-config\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-web-config\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698695 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fdc27892-e7ef-41a2-b3ea-4a96d0753927-config-out\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698845 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.698930 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.698846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.699198 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.699174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fdc27892-e7ef-41a2-b3ea-4a96d0753927-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.699278 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.699234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fdc27892-e7ef-41a2-b3ea-4a96d0753927-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.700187 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.700164    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.700556 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.700535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.700891 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.700871    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.701082 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.701065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.701314 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.701297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fdc27892-e7ef-41a2-b3ea-4a96d0753927-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.703120 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.703081    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lffjg\" (UniqueName: \"kubernetes.io/projected/fdc27892-e7ef-41a2-b3ea-4a96d0753927-kube-api-access-lffjg\") pod \"prometheus-k8s-0\" (UID: \"fdc27892-e7ef-41a2-b3ea-4a96d0753927\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.824095 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.824046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:17:49.959561 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:49.959498    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 29 07:17:49.962833 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:17:49.962806    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc27892_e7ef_41a2_b3ea_4a96d0753927.slice/crio-4126104a8bfb481c4c885b10d553e8909ca6a214c98f046bcbfbf8bac8495bb6 WatchSource:0}: Error finding container 4126104a8bfb481c4c885b10d553e8909ca6a214c98f046bcbfbf8bac8495bb6: Status 404 returned error can't find the container with id 4126104a8bfb481c4c885b10d553e8909ca6a214c98f046bcbfbf8bac8495bb6
Jul 29 07:17:50.458587 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:50.458555    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdc27892-e7ef-41a2-b3ea-4a96d0753927" containerID="8e69d2ca99485eac1597956fea0b8c7de80e406b33f3dee1d461e8e722f68959" exitCode=0
Jul 29 07:17:50.458756 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:50.458600    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerDied","Data":"8e69d2ca99485eac1597956fea0b8c7de80e406b33f3dee1d461e8e722f68959"}
Jul 29 07:17:50.458756 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:50.458694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"4126104a8bfb481c4c885b10d553e8909ca6a214c98f046bcbfbf8bac8495bb6"}
Jul 29 07:17:50.790192 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:50.790162    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5" path="/var/lib/kubelet/pods/43c86aa9-1c07-4ef0-b0c4-72f9ae077aa5/volumes"
Jul 29 07:17:51.466936 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.466891    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"ca7409cd8bac3b674e2c1185f094bc82c135f08673072434cef6894f1e1fd781"}
Jul 29 07:17:51.467329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.466977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"e264f43ca88f3ad44bad72db4eb205b5143e3576e7f867ca6aec449a56fd2ee4"}
Jul 29 07:17:51.467329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.467006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"1a0a266b624eeb39dfa1871b19c9cf600130438cc52ce65ee520bbb44345a20b"}
Jul 29 07:17:51.467329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.467031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"f3210c1a89a4d123d089d3d7d4ad164bb10027d725be05c2d9b3649e7720a9d8"}
Jul 29 07:17:51.467329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.467053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"52486977cb7d5bb9bddecf778e7c50384b83d3011dbcfd8b716e66c291041e88"}
Jul 29 07:17:51.467329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.467078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fdc27892-e7ef-41a2-b3ea-4a96d0753927","Type":"ContainerStarted","Data":"1d0b009196f38f0e57b118490d9b77743a89a5858137fd9b7325fc593df90047"}
Jul 29 07:17:51.496138 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:51.496089    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.473745503 podStartE2EDuration="2.496071314s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.325811ms" isStatefulPod=false podCreationTimestamp="2026-07-29 07:17:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:17:51.493675025 +0000 UTC m=+153.241740893" watchObservedRunningTime="2026-07-29 07:17:51.496071314 +0000 UTC m=+153.244137155"
Jul 29 07:17:54.824568 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:17:54.824537    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:18:47.156210 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.156175    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"]
Jul 29 07:18:47.158244 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.158229    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.161739 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.161719    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 07:18:47.161874 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.161738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-9ph4h\""
Jul 29 07:18:47.162839 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.162812    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 07:18:47.176720 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.176700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"]
Jul 29 07:18:47.244006 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.243973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl55k\" (UniqueName: \"kubernetes.io/projected/5bbdd85b-2bec-4a39-9c27-5bf51a5aced9-kube-api-access-wl55k\") pod \"cert-manager-cainjector-559cdd8c87-kjxvz\" (UID: \"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.244137 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.244036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bbdd85b-2bec-4a39-9c27-5bf51a5aced9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kjxvz\" (UID: \"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.345418 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.345375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bbdd85b-2bec-4a39-9c27-5bf51a5aced9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kjxvz\" (UID: \"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.345569 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.345456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl55k\" (UniqueName: \"kubernetes.io/projected/5bbdd85b-2bec-4a39-9c27-5bf51a5aced9-kube-api-access-wl55k\") pod \"cert-manager-cainjector-559cdd8c87-kjxvz\" (UID: \"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.354178 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.354142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5bbdd85b-2bec-4a39-9c27-5bf51a5aced9-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kjxvz\" (UID: \"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.354285 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.354220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl55k\" (UniqueName: \"kubernetes.io/projected/5bbdd85b-2bec-4a39-9c27-5bf51a5aced9-kube-api-access-wl55k\") pod \"cert-manager-cainjector-559cdd8c87-kjxvz\" (UID: \"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.479347 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.479261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"
Jul 29 07:18:47.601352 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.601321    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz"]
Jul 29 07:18:47.604254 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:18:47.604224    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5bbdd85b_2bec_4a39_9c27_5bf51a5aced9.slice/crio-ce61dc26fd2afa5e5e7989d42ddb349ef622478ac2045e2c959b5fb73d48b5c0 WatchSource:0}: Error finding container ce61dc26fd2afa5e5e7989d42ddb349ef622478ac2045e2c959b5fb73d48b5c0: Status 404 returned error can't find the container with id ce61dc26fd2afa5e5e7989d42ddb349ef622478ac2045e2c959b5fb73d48b5c0
Jul 29 07:18:47.661432 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:47.661404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz" event={"ID":"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9","Type":"ContainerStarted","Data":"ce61dc26fd2afa5e5e7989d42ddb349ef622478ac2045e2c959b5fb73d48b5c0"}
Jul 29 07:18:49.825260 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:49.825212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:18:49.842326 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:49.842293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:18:50.675127 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:50.675088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz" event={"ID":"5bbdd85b-2bec-4a39-9c27-5bf51a5aced9","Type":"ContainerStarted","Data":"eaad9756f02ccd09300346a12d243087e540afb6e68099efb975658617ab7fa1"}
Jul 29 07:18:50.692617 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:50.692568    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kjxvz" podStartSLOduration=0.694910879 podStartE2EDuration="3.692553858s" totalImagesPullingTime="2.997642979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:18:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:18:50.691168079 +0000 UTC m=+212.439233921" watchObservedRunningTime="2026-07-29 07:18:50.692553858 +0000 UTC m=+212.440619699"
Jul 29 07:18:50.695462 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:50.695435    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 29 07:18:58.382361 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.382327    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"]
Jul 29 07:18:58.384602 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.384586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.387446 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.387425    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 07:18:58.387568 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.387425    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-sltcn\""
Jul 29 07:18:58.388709 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.388691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 07:18:58.394297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.394275    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"]
Jul 29 07:18:58.533017 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.532983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw2zq\" (UniqueName: \"kubernetes.io/projected/680a2962-a879-4443-96ed-009e74a88a29-kube-api-access-mw2zq\") pod \"openshift-lws-operator-fd8ccff4c-4mskb\" (UID: \"680a2962-a879-4443-96ed-009e74a88a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.533184 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.533088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/680a2962-a879-4443-96ed-009e74a88a29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4mskb\" (UID: \"680a2962-a879-4443-96ed-009e74a88a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.633765 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.633681    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/680a2962-a879-4443-96ed-009e74a88a29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4mskb\" (UID: \"680a2962-a879-4443-96ed-009e74a88a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.633765 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.633729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw2zq\" (UniqueName: \"kubernetes.io/projected/680a2962-a879-4443-96ed-009e74a88a29-kube-api-access-mw2zq\") pod \"openshift-lws-operator-fd8ccff4c-4mskb\" (UID: \"680a2962-a879-4443-96ed-009e74a88a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.634127 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.634104    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/680a2962-a879-4443-96ed-009e74a88a29-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4mskb\" (UID: \"680a2962-a879-4443-96ed-009e74a88a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.643510 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.643483    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw2zq\" (UniqueName: \"kubernetes.io/projected/680a2962-a879-4443-96ed-009e74a88a29-kube-api-access-mw2zq\") pod \"openshift-lws-operator-fd8ccff4c-4mskb\" (UID: \"680a2962-a879-4443-96ed-009e74a88a29\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.693157 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.693132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"
Jul 29 07:18:58.819967 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:58.819939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb"]
Jul 29 07:18:58.823208 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:18:58.823181    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod680a2962_a879_4443_96ed_009e74a88a29.slice/crio-d1b2216ce9299ae71df3528ffb09d1593caa8d24f6880cd47a3756afebbff17c WatchSource:0}: Error finding container d1b2216ce9299ae71df3528ffb09d1593caa8d24f6880cd47a3756afebbff17c: Status 404 returned error can't find the container with id d1b2216ce9299ae71df3528ffb09d1593caa8d24f6880cd47a3756afebbff17c
Jul 29 07:18:59.706400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:18:59.706364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb" event={"ID":"680a2962-a879-4443-96ed-009e74a88a29","Type":"ContainerStarted","Data":"d1b2216ce9299ae71df3528ffb09d1593caa8d24f6880cd47a3756afebbff17c"}
Jul 29 07:19:01.714996 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:01.714957    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb" event={"ID":"680a2962-a879-4443-96ed-009e74a88a29","Type":"ContainerStarted","Data":"871283e60d73f87aaa39423c58659d346cc658060312e3a30db8d0cfd74359c2"}
Jul 29 07:19:01.732726 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:01.732680    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4mskb" podStartSLOduration=1.438698762 podStartE2EDuration="3.732663147s" totalImagesPullingTime="2.293964385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:19:01.732355771 +0000 UTC m=+223.480421626" watchObservedRunningTime="2026-07-29 07:19:01.732663147 +0000 UTC m=+223.480728988"
Jul 29 07:19:19.037486 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.037451    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"]
Jul 29 07:19:19.040643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.040620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.050720 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.050696    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 29 07:19:19.050835 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.050737    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 29 07:19:19.050835 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.050754    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g4gdz\""
Jul 29 07:19:19.050835 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.050761    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 29 07:19:19.057221 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.057202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"]
Jul 29 07:19:19.100593 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.100553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4pdj\" (UniqueName: \"kubernetes.io/projected/33e73cf7-7ec4-484a-b1a4-36679b1e71da-kube-api-access-d4pdj\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.100593 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.100595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33e73cf7-7ec4-484a-b1a4-36679b1e71da-metrics-cert\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.100820 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.100618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33e73cf7-7ec4-484a-b1a4-36679b1e71da-cert\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.100820 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.100696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/33e73cf7-7ec4-484a-b1a4-36679b1e71da-manager-config\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.202027 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.201993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4pdj\" (UniqueName: \"kubernetes.io/projected/33e73cf7-7ec4-484a-b1a4-36679b1e71da-kube-api-access-d4pdj\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.202245 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.202032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33e73cf7-7ec4-484a-b1a4-36679b1e71da-metrics-cert\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.202245 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.202056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33e73cf7-7ec4-484a-b1a4-36679b1e71da-cert\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.202245 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.202081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/33e73cf7-7ec4-484a-b1a4-36679b1e71da-manager-config\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.202792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.202764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/33e73cf7-7ec4-484a-b1a4-36679b1e71da-manager-config\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.204493 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.204464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/33e73cf7-7ec4-484a-b1a4-36679b1e71da-metrics-cert\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.204601 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.204556    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/33e73cf7-7ec4-484a-b1a4-36679b1e71da-cert\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.216014 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.215986    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4pdj\" (UniqueName: \"kubernetes.io/projected/33e73cf7-7ec4-484a-b1a4-36679b1e71da-kube-api-access-d4pdj\") pod \"lws-controller-manager-d85576cc8-zz7b6\" (UID: \"33e73cf7-7ec4-484a-b1a4-36679b1e71da\") " pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.249487 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.249447    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"]
Jul 29 07:19:19.256301 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.256273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.259571 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.259542    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 07:19:19.259704 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.259580    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 07:19:19.259704 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.259553    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 07:19:19.259704 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.259614    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 07:19:19.260507 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.260493    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sjl6m\""
Jul 29 07:19:19.275016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.274994    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"]
Jul 29 07:19:19.302719 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.302642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.302719 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.302679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.302889 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.302760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-manifests\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.302889 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.302786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-tmp\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.302889 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.302818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-689zj\" (UniqueName: \"kubernetes.io/projected/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-kube-api-access-689zj\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.349690 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.349650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:19.403967 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.403930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-manifests\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.404114 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.403980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-tmp\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.404114 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.404017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-689zj\" (UniqueName: \"kubernetes.io/projected/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-kube-api-access-689zj\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.404114 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.404066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.404114 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.404102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.404656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.404375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-manifests\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.407134 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.407111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-tmp\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.407702 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.407662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.408110 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.408092    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.413812 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.413776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-689zj\" (UniqueName: \"kubernetes.io/projected/230995ba-b9fb-48d8-b0cc-9345f8b60a3e-kube-api-access-689zj\") pod \"opendatahub-operator-controller-manager-6d6c966467-m8snl\" (UID: \"230995ba-b9fb-48d8-b0cc-9345f8b60a3e\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.483196 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.483163    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"]
Jul 29 07:19:19.485737 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:19:19.485715    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e73cf7_7ec4_484a_b1a4_36679b1e71da.slice/crio-4adaf4a4a654e7711a6ecce9c4bc8221c52aeff5e690840187a10a2b951f132d WatchSource:0}: Error finding container 4adaf4a4a654e7711a6ecce9c4bc8221c52aeff5e690840187a10a2b951f132d: Status 404 returned error can't find the container with id 4adaf4a4a654e7711a6ecce9c4bc8221c52aeff5e690840187a10a2b951f132d
Jul 29 07:19:19.566295 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.566216    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:19.701195 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.701165    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"]
Jul 29 07:19:19.704117 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:19:19.704090    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230995ba_b9fb_48d8_b0cc_9345f8b60a3e.slice/crio-4d14cf31a57452294ae627c7b2ff7f61b8ee1e0eafeedc601a4a584c6497d78c WatchSource:0}: Error finding container 4d14cf31a57452294ae627c7b2ff7f61b8ee1e0eafeedc601a4a584c6497d78c: Status 404 returned error can't find the container with id 4d14cf31a57452294ae627c7b2ff7f61b8ee1e0eafeedc601a4a584c6497d78c
Jul 29 07:19:19.782233 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.782192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6" event={"ID":"33e73cf7-7ec4-484a-b1a4-36679b1e71da","Type":"ContainerStarted","Data":"4adaf4a4a654e7711a6ecce9c4bc8221c52aeff5e690840187a10a2b951f132d"}
Jul 29 07:19:19.783350 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:19.783328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl" event={"ID":"230995ba-b9fb-48d8-b0cc-9345f8b60a3e","Type":"ContainerStarted","Data":"4d14cf31a57452294ae627c7b2ff7f61b8ee1e0eafeedc601a4a584c6497d78c"}
Jul 29 07:19:23.803743 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:23.803709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl" event={"ID":"230995ba-b9fb-48d8-b0cc-9345f8b60a3e","Type":"ContainerStarted","Data":"238c5ca6eff97330b23865650654e2c1b001bb80ad10a5cc9a1983b155925bb8"}
Jul 29 07:19:23.805209 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:23.805188    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6" event={"ID":"33e73cf7-7ec4-484a-b1a4-36679b1e71da","Type":"ContainerStarted","Data":"5dfe74e36dc4f3a28964e361baff0caf48dec6604e74d893e249738e9305060f"}
Jul 29 07:19:23.805329 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:23.805313    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:23.846588 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:23.846544    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6" podStartSLOduration=1.248718884 podStartE2EDuration="4.846529899s" totalImagesPullingTime="3.597811015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:19:23.845476878 +0000 UTC m=+245.593542737" watchObservedRunningTime="2026-07-29 07:19:23.846529899 +0000 UTC m=+245.594595741"
Jul 29 07:19:24.810140 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:24.810098    2577 generic.go:356] "Generic (PLEG): container finished" podID="230995ba-b9fb-48d8-b0cc-9345f8b60a3e" containerID="238c5ca6eff97330b23865650654e2c1b001bb80ad10a5cc9a1983b155925bb8" exitCode=0
Jul 29 07:19:24.810604 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:24.810192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl" event={"ID":"230995ba-b9fb-48d8-b0cc-9345f8b60a3e","Type":"ContainerDied","Data":"238c5ca6eff97330b23865650654e2c1b001bb80ad10a5cc9a1983b155925bb8"}
Jul 29 07:19:25.815983 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:25.815946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl" event={"ID":"230995ba-b9fb-48d8-b0cc-9345f8b60a3e","Type":"ContainerStarted","Data":"7c1200fd3469d523b500e989bb72b2ab52ad7261b5171cf796cb968d0da4d65b"}
Jul 29 07:19:25.816385 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:25.816081    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:25.838075 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:25.838025    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl" podStartSLOduration=1.967831946 podStartE2EDuration="6.838007916s" totalImagesPullingTime="3.828540152s" totalInitContainerRuntime="1.041635818s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:19:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:19:25.83656738 +0000 UTC m=+247.584633242" watchObservedRunningTime="2026-07-29 07:19:25.838007916 +0000 UTC m=+247.586073758"
Jul 29 07:19:34.812595 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:34.812565    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d85576cc8-zz7b6"
Jul 29 07:19:36.823292 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:36.823263    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6c966467-m8snl"
Jul 29 07:19:38.963043 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.963009    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"]
Jul 29 07:19:38.970271 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.970251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:38.973723 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.973697    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 07:19:38.973858 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.973775    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jfn7q\""
Jul 29 07:19:38.973858 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.973804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 07:19:38.974006 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.973869    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 07:19:38.974006 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.973886    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 07:19:38.980640 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:38.980620    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"]
Jul 29 07:19:39.067671 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.067626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tmp\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.067856 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.067742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr6h8\" (UniqueName: \"kubernetes.io/projected/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-kube-api-access-wr6h8\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.067856 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.067818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tls-certs\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.169130 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.169089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tls-certs\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.169130 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.169136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tmp\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.169336 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.169177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr6h8\" (UniqueName: \"kubernetes.io/projected/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-kube-api-access-wr6h8\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.169336 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:19:39.169258    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 29 07:19:39.169407 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:19:39.169338    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tls-certs podName:741a0e9a-3ee1-499b-ab50-65c1bc83c52c nodeName:}" failed. No retries permitted until 2026-07-29 07:19:39.669317715 +0000 UTC m=+261.417383535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tls-certs") pod "kube-auth-proxy-6dd888fd86-sn4q9" (UID: "741a0e9a-3ee1-499b-ab50-65c1bc83c52c") : secret "kube-auth-proxy-tls" not found
Jul 29 07:19:39.171480 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.171453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tmp\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.185813 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.185785    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr6h8\" (UniqueName: \"kubernetes.io/projected/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-kube-api-access-wr6h8\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.674630 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.674589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tls-certs\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.677064 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.677043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/741a0e9a-3ee1-499b-ab50-65c1bc83c52c-tls-certs\") pod \"kube-auth-proxy-6dd888fd86-sn4q9\" (UID: \"741a0e9a-3ee1-499b-ab50-65c1bc83c52c\") " pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:39.881088 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:39.880978    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"
Jul 29 07:19:40.025222 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:40.025193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9"]
Jul 29 07:19:40.027207 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:19:40.027182    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741a0e9a_3ee1_499b_ab50_65c1bc83c52c.slice/crio-488d6df8b803bf2eab9de5415a3b1f4be73d76456b7f939a720b94d0066a6d25 WatchSource:0}: Error finding container 488d6df8b803bf2eab9de5415a3b1f4be73d76456b7f939a720b94d0066a6d25: Status 404 returned error can't find the container with id 488d6df8b803bf2eab9de5415a3b1f4be73d76456b7f939a720b94d0066a6d25
Jul 29 07:19:40.877651 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:40.877569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9" event={"ID":"741a0e9a-3ee1-499b-ab50-65c1bc83c52c","Type":"ContainerStarted","Data":"488d6df8b803bf2eab9de5415a3b1f4be73d76456b7f939a720b94d0066a6d25"}
Jul 29 07:19:42.887164 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:42.887127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9" event={"ID":"741a0e9a-3ee1-499b-ab50-65c1bc83c52c","Type":"ContainerStarted","Data":"07fdf41cab45d49197c46bf32087a66f80170dbf352a2761978668ad117034a1"}
Jul 29 07:19:42.904068 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:42.904013    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6dd888fd86-sn4q9" podStartSLOduration=2.590829119 podStartE2EDuration="4.903997275s" totalImagesPullingTime="2.313168156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:19:42.903046192 +0000 UTC m=+264.651112035" watchObservedRunningTime="2026-07-29 07:19:42.903997275 +0000 UTC m=+264.652063117"
Jul 29 07:19:49.803781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.803744    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"]
Jul 29 07:19:49.806141 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.806124    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:49.816656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.816631    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-pm6n2\""
Jul 29 07:19:49.817339 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.817323    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 07:19:49.838233 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.838195    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"]
Jul 29 07:19:49.969557 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.969524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5p9\" (UniqueName: \"kubernetes.io/projected/4171bc0f-d2e1-4bff-a32f-29091b80541d-kube-api-access-jc5p9\") pod \"kserve-controller-manager-d6f4b65b8-qqdvk\" (UID: \"4171bc0f-d2e1-4bff-a32f-29091b80541d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:49.969730 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:49.969573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4171bc0f-d2e1-4bff-a32f-29091b80541d-cert\") pod \"kserve-controller-manager-d6f4b65b8-qqdvk\" (UID: \"4171bc0f-d2e1-4bff-a32f-29091b80541d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:50.070812 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.070728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5p9\" (UniqueName: \"kubernetes.io/projected/4171bc0f-d2e1-4bff-a32f-29091b80541d-kube-api-access-jc5p9\") pod \"kserve-controller-manager-d6f4b65b8-qqdvk\" (UID: \"4171bc0f-d2e1-4bff-a32f-29091b80541d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:50.070812 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.070776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4171bc0f-d2e1-4bff-a32f-29091b80541d-cert\") pod \"kserve-controller-manager-d6f4b65b8-qqdvk\" (UID: \"4171bc0f-d2e1-4bff-a32f-29091b80541d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:50.073295 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.073272    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4171bc0f-d2e1-4bff-a32f-29091b80541d-cert\") pod \"kserve-controller-manager-d6f4b65b8-qqdvk\" (UID: \"4171bc0f-d2e1-4bff-a32f-29091b80541d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:50.078729 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.078702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5p9\" (UniqueName: \"kubernetes.io/projected/4171bc0f-d2e1-4bff-a32f-29091b80541d-kube-api-access-jc5p9\") pod \"kserve-controller-manager-d6f4b65b8-qqdvk\" (UID: \"4171bc0f-d2e1-4bff-a32f-29091b80541d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:50.115655 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.115625    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:50.257746 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.257711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"]
Jul 29 07:19:50.261780 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:19:50.261752    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4171bc0f_d2e1_4bff_a32f_29091b80541d.slice/crio-5e6558300613a6b7576e979e6426376818b696dff9723acffca27895c6b4010e WatchSource:0}: Error finding container 5e6558300613a6b7576e979e6426376818b696dff9723acffca27895c6b4010e: Status 404 returned error can't find the container with id 5e6558300613a6b7576e979e6426376818b696dff9723acffca27895c6b4010e
Jul 29 07:19:50.921040 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:50.920998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk" event={"ID":"4171bc0f-d2e1-4bff-a32f-29091b80541d","Type":"ContainerStarted","Data":"5e6558300613a6b7576e979e6426376818b696dff9723acffca27895c6b4010e"}
Jul 29 07:19:52.936113 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:52.936074    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk" event={"ID":"4171bc0f-d2e1-4bff-a32f-29091b80541d","Type":"ContainerStarted","Data":"5abb81734c35f7c6da5ad0b21473b891127e6af083491887f04bb3bec32964a2"}
Jul 29 07:19:52.936509 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:52.936169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:19:52.958382 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:19:52.958333    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk" podStartSLOduration=1.464257184 podStartE2EDuration="3.958318592s" totalImagesPullingTime="2.494061408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:19:52.956308844 +0000 UTC m=+274.704374686" watchObservedRunningTime="2026-07-29 07:19:52.958318592 +0000 UTC m=+274.706384433"
Jul 29 07:20:18.602071 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:20:18.602029    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 07:20:23.945612 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:20:23.945582    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qqdvk"
Jul 29 07:21:34.846235 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.846202    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"]
Jul 29 07:21:34.849705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.849682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:34.852313 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.852288    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 07:21:34.852432 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.852398    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2l2dr\""
Jul 29 07:21:34.852490 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.852447    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 07:21:34.853616 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.853595    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 07:21:34.853705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.853604    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 07:21:34.858357 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.858337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"]
Jul 29 07:21:34.936727 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.936692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:34.936943 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.936809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:34.936943 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:34.936861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vq4r\" (UniqueName: \"kubernetes.io/projected/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-kube-api-access-2vq4r\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.037613 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.037579    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.037803 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.037665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.037803 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.037705    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vq4r\" (UniqueName: \"kubernetes.io/projected/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-kube-api-access-2vq4r\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.038261 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.038240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.040010 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.039981    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.046371 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.046350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vq4r\" (UniqueName: \"kubernetes.io/projected/97ed7bde-5e79-428a-abb7-c8fbe2a5ea15-kube-api-access-2vq4r\") pod \"kuadrant-console-plugin-66b5774985-cf4kx\" (UID: \"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.158824 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.158777    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"
Jul 29 07:21:35.294624 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.294591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx"]
Jul 29 07:21:35.297866 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:21:35.297840    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97ed7bde_5e79_428a_abb7_c8fbe2a5ea15.slice/crio-d4fbe73fae382b3715f81990f9b4c7df219fad1181c259bc8b8c3b6bc0fb2ddd WatchSource:0}: Error finding container d4fbe73fae382b3715f81990f9b4c7df219fad1181c259bc8b8c3b6bc0fb2ddd: Status 404 returned error can't find the container with id d4fbe73fae382b3715f81990f9b4c7df219fad1181c259bc8b8c3b6bc0fb2ddd
Jul 29 07:21:35.299519 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.299503    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 07:21:35.359084 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:35.359047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx" event={"ID":"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15","Type":"ContainerStarted","Data":"d4fbe73fae382b3715f81990f9b4c7df219fad1181c259bc8b8c3b6bc0fb2ddd"}
Jul 29 07:21:56.469079 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:56.469037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx" event={"ID":"97ed7bde-5e79-428a-abb7-c8fbe2a5ea15","Type":"ContainerStarted","Data":"4625bfdd9b3c5f40ed2c9c8540f305bbc26dde164239e6ce69e2d32b8464787e"}
Jul 29 07:21:56.488373 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:21:56.488321    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-cf4kx" podStartSLOduration=1.45386093 podStartE2EDuration="22.488305914s" totalImagesPullingTime="21.034444984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:21:56.486759022 +0000 UTC m=+398.234824868" watchObservedRunningTime="2026-07-29 07:21:56.488305914 +0000 UTC m=+398.236371756"
Jul 29 07:22:35.342887 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.342850    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:22:35.363781 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.363747    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:22:35.363971 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.363857    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.366782 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.366758    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 07:22:35.375566 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.375540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:22:35.456378 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.456349    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wdgl\" (UniqueName: \"kubernetes.io/projected/6fb2d089-525f-410e-a6f2-972a5f1f2f8e-kube-api-access-6wdgl\") pod \"limitador-limitador-54d6d988d5-wvjfq\" (UID: \"6fb2d089-525f-410e-a6f2-972a5f1f2f8e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.456540 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.456391    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6fb2d089-525f-410e-a6f2-972a5f1f2f8e-config-file\") pod \"limitador-limitador-54d6d988d5-wvjfq\" (UID: \"6fb2d089-525f-410e-a6f2-972a5f1f2f8e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.557325 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.557294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wdgl\" (UniqueName: \"kubernetes.io/projected/6fb2d089-525f-410e-a6f2-972a5f1f2f8e-kube-api-access-6wdgl\") pod \"limitador-limitador-54d6d988d5-wvjfq\" (UID: \"6fb2d089-525f-410e-a6f2-972a5f1f2f8e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.557469 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.557330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6fb2d089-525f-410e-a6f2-972a5f1f2f8e-config-file\") pod \"limitador-limitador-54d6d988d5-wvjfq\" (UID: \"6fb2d089-525f-410e-a6f2-972a5f1f2f8e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.557955 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.557936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6fb2d089-525f-410e-a6f2-972a5f1f2f8e-config-file\") pod \"limitador-limitador-54d6d988d5-wvjfq\" (UID: \"6fb2d089-525f-410e-a6f2-972a5f1f2f8e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.565989 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.565968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wdgl\" (UniqueName: \"kubernetes.io/projected/6fb2d089-525f-410e-a6f2-972a5f1f2f8e-kube-api-access-6wdgl\") pod \"limitador-limitador-54d6d988d5-wvjfq\" (UID: \"6fb2d089-525f-410e-a6f2-972a5f1f2f8e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.673656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.673628    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:35.752970 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.752932    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-2glxp"]
Jul 29 07:22:35.755927 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.755826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:35.758840 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.758818    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qw5qb\""
Jul 29 07:22:35.763361 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.763306    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2glxp"]
Jul 29 07:22:35.807215 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.807180    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:22:35.810762 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:22:35.810730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb2d089_525f_410e_a6f2_972a5f1f2f8e.slice/crio-983111ea9ad2d9cbf10e421fab64e1e853e1de61f94115bb8633bc081ba0c008 WatchSource:0}: Error finding container 983111ea9ad2d9cbf10e421fab64e1e853e1de61f94115bb8633bc081ba0c008: Status 404 returned error can't find the container with id 983111ea9ad2d9cbf10e421fab64e1e853e1de61f94115bb8633bc081ba0c008
Jul 29 07:22:35.859557 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.859513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttcf\" (UniqueName: \"kubernetes.io/projected/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114-kube-api-access-lttcf\") pod \"authorino-58cd955675-2glxp\" (UID: \"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114\") " pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:35.960816 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.960730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttcf\" (UniqueName: \"kubernetes.io/projected/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114-kube-api-access-lttcf\") pod \"authorino-58cd955675-2glxp\" (UID: \"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114\") " pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:35.971650 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:35.971618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttcf\" (UniqueName: \"kubernetes.io/projected/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114-kube-api-access-lttcf\") pod \"authorino-58cd955675-2glxp\" (UID: \"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114\") " pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:36.068547 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:36.068514    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:36.192249 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:36.192219    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-2glxp"]
Jul 29 07:22:36.195142 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:22:36.195110    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a2dfbd0_4596_4cd0_ad80_0cd9d6501114.slice/crio-f5eb3b381f12e878fe79ba3ef255a1efd87008ac317dd910893fe9573d205b8c WatchSource:0}: Error finding container f5eb3b381f12e878fe79ba3ef255a1efd87008ac317dd910893fe9573d205b8c: Status 404 returned error can't find the container with id f5eb3b381f12e878fe79ba3ef255a1efd87008ac317dd910893fe9573d205b8c
Jul 29 07:22:36.635519 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:36.635474    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2glxp" event={"ID":"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114","Type":"ContainerStarted","Data":"f5eb3b381f12e878fe79ba3ef255a1efd87008ac317dd910893fe9573d205b8c"}
Jul 29 07:22:36.636969 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:36.636943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq" event={"ID":"6fb2d089-525f-410e-a6f2-972a5f1f2f8e","Type":"ContainerStarted","Data":"983111ea9ad2d9cbf10e421fab64e1e853e1de61f94115bb8633bc081ba0c008"}
Jul 29 07:22:40.659363 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:40.659328    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq" event={"ID":"6fb2d089-525f-410e-a6f2-972a5f1f2f8e","Type":"ContainerStarted","Data":"2317184529b4ebb9c9ad05082c86bba31b6077964f27534c4bb704f36fbdfc7a"}
Jul 29 07:22:40.659825 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:40.659400    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:22:40.660812 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:40.660789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2glxp" event={"ID":"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114","Type":"ContainerStarted","Data":"99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b"}
Jul 29 07:22:40.682487 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:40.682442    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq" podStartSLOduration=1.336014441 podStartE2EDuration="5.68242687s" totalImagesPullingTime="4.346412429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:22:40.676208782 +0000 UTC m=+442.424274634" watchObservedRunningTime="2026-07-29 07:22:40.68242687 +0000 UTC m=+442.430492711"
Jul 29 07:22:40.692045 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:40.691995    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-2glxp" podStartSLOduration=1.722413279 podStartE2EDuration="5.691975833s" totalImagesPullingTime="3.969562554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:22:40.691436894 +0000 UTC m=+442.439502733" watchObservedRunningTime="2026-07-29 07:22:40.691975833 +0000 UTC m=+442.440041677"
Jul 29 07:22:41.368059 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:41.368025    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2glxp"]
Jul 29 07:22:42.670124 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:42.670085    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-2glxp" podUID="7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" containerName="authorino" containerID="cri-o://99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b" gracePeriod=30
Jul 29 07:22:42.914837 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:42.914811    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:43.024118 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.024021    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lttcf\" (UniqueName: \"kubernetes.io/projected/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114-kube-api-access-lttcf\") pod \"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114\" (UID: \"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114\") "
Jul 29 07:22:43.026140 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.026114    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114-kube-api-access-lttcf" (OuterVolumeSpecName: "kube-api-access-lttcf") pod "7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" (UID: "7a2dfbd0-4596-4cd0-ad80-0cd9d6501114"). InnerVolumeSpecName "kube-api-access-lttcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:22:43.125506 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.125467    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lttcf\" (UniqueName: \"kubernetes.io/projected/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114-kube-api-access-lttcf\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:22:43.675575 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.675542    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" containerID="99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b" exitCode=0
Jul 29 07:22:43.676046 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.675582    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-2glxp"
Jul 29 07:22:43.676046 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.675628    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2glxp" event={"ID":"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114","Type":"ContainerDied","Data":"99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b"}
Jul 29 07:22:43.676046 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.675666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-2glxp" event={"ID":"7a2dfbd0-4596-4cd0-ad80-0cd9d6501114","Type":"ContainerDied","Data":"f5eb3b381f12e878fe79ba3ef255a1efd87008ac317dd910893fe9573d205b8c"}
Jul 29 07:22:43.676046 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.675683    2577 scope.go:117] "RemoveContainer" containerID="99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b"
Jul 29 07:22:43.683777 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.683762    2577 scope.go:117] "RemoveContainer" containerID="99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b"
Jul 29 07:22:43.684027 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:22:43.684008    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b\": container with ID starting with 99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b not found: ID does not exist" containerID="99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b"
Jul 29 07:22:43.684098 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.684033    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b"} err="failed to get container status \"99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b\": rpc error: code = NotFound desc = could not find container \"99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b\": container with ID starting with 99e94fe072b351d98216293c37bf5197ab5fa58c51b8f3be2e0b9ed6ea2a300b not found: ID does not exist"
Jul 29 07:22:43.695749 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.695727    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-2glxp"]
Jul 29 07:22:43.699274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:43.699253    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-2glxp"]
Jul 29 07:22:44.789112 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:44.789072    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" path="/var/lib/kubelet/pods/7a2dfbd0-4596-4cd0-ad80-0cd9d6501114/volumes"
Jul 29 07:22:51.667325 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:22:51.667294    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"
Jul 29 07:23:55.419368 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:23:55.419332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:24:10.718834 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:10.718762    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:24:19.256082 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:19.256026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:24:43.275710 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.275677    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"]
Jul 29 07:24:43.276172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.276002    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" containerName="authorino"
Jul 29 07:24:43.276172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.276013    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" containerName="authorino"
Jul 29 07:24:43.276172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.276077    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a2dfbd0-4596-4cd0-ad80-0cd9d6501114" containerName="authorino"
Jul 29 07:24:43.278160 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.278135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.282248 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.282224    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4s7sl\""
Jul 29 07:24:43.282380 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.282226    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 07:24:43.282380 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.282268    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 07:24:43.282380 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.282268    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 07:24:43.296738 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.296715    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"]
Jul 29 07:24:43.320715 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.320690    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.320804 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.320722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.320804 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.320742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.320876 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.320816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.320876 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.320855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06373343-d7aa-48d8-a2f9-464b4fe182bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.320975 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.320876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q78k8\" (UniqueName: \"kubernetes.io/projected/06373343-d7aa-48d8-a2f9-464b4fe182bc-kube-api-access-q78k8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.421956 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.421896    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.421956 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.421959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422192 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.421981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422192 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.422007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422192 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.422029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06373343-d7aa-48d8-a2f9-464b4fe182bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422192 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.422045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q78k8\" (UniqueName: \"kubernetes.io/projected/06373343-d7aa-48d8-a2f9-464b4fe182bc-kube-api-access-q78k8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422472 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.422450    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422558 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.422458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.422558 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.422494    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.424727 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.424703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06373343-d7aa-48d8-a2f9-464b4fe182bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.424982 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.424965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06373343-d7aa-48d8-a2f9-464b4fe182bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.430915 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.430875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q78k8\" (UniqueName: \"kubernetes.io/projected/06373343-d7aa-48d8-a2f9-464b4fe182bc-kube-api-access-q78k8\") pod \"e2e-trlp-test-simulated-kserve-598466548d-xgmzf\" (UID: \"06373343-d7aa-48d8-a2f9-464b4fe182bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.588618 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.588516    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:43.723643 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:43.723618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"]
Jul 29 07:24:43.726480 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:24:43.726451    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06373343_d7aa_48d8_a2f9_464b4fe182bc.slice/crio-e6281eb9b66116cedb841cc7c975f6ab5996cf3ec008d3d022fd66d6e9849186 WatchSource:0}: Error finding container e6281eb9b66116cedb841cc7c975f6ab5996cf3ec008d3d022fd66d6e9849186: Status 404 returned error can't find the container with id e6281eb9b66116cedb841cc7c975f6ab5996cf3ec008d3d022fd66d6e9849186
Jul 29 07:24:44.184054 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:44.184014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf" event={"ID":"06373343-d7aa-48d8-a2f9-464b4fe182bc","Type":"ContainerStarted","Data":"e6281eb9b66116cedb841cc7c975f6ab5996cf3ec008d3d022fd66d6e9849186"}
Jul 29 07:24:44.598503 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:44.598421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:24:45.189245 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:45.189213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf" event={"ID":"06373343-d7aa-48d8-a2f9-464b4fe182bc","Type":"ContainerStarted","Data":"19c4af1844af5c34d7914ee45b63ff297430c6f83732d17e41d117bd42c6f71c"}
Jul 29 07:24:45.209829 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:45.209770    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf" podStartSLOduration=0.960629095 podStartE2EDuration="2.209753855s" totalImagesPullingTime="1.24912476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:24:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:24:45.208636925 +0000 UTC m=+566.956702768" watchObservedRunningTime="2026-07-29 07:24:45.209753855 +0000 UTC m=+566.957819717"
Jul 29 07:24:53.589395 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:53.589342    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:53.589793 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:53.589429    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:53.601841 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:53.601814    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:24:54.242459 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:24:54.242427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-xgmzf"
Jul 29 07:25:01.026250 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:01.026214    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:25:12.425456 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.425422    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"]
Jul 29 07:25:12.427778 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.427761    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.431088 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.431066    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 07:25:12.444825 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.444806    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"]
Jul 29 07:25:12.472690 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.472654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.472859 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.472707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c666h\" (UniqueName: \"kubernetes.io/projected/68f91e4a-d089-4ce5-badf-dc1bef143add-kube-api-access-c666h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.472859 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.472773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.472859 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.472798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68f91e4a-d089-4ce5-badf-dc1bef143add-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.472859 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.472836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.472859 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.472854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574146 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574111    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574146 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574149    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c666h\" (UniqueName: \"kubernetes.io/projected/68f91e4a-d089-4ce5-badf-dc1bef143add-kube-api-access-c666h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574195    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68f91e4a-d089-4ce5-badf-dc1bef143add-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574400 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574318    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574602 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574657 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.574713 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.574675    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.576652 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.576625    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68f91e4a-d089-4ce5-badf-dc1bef143add-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.576879 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.576857    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68f91e4a-d089-4ce5-badf-dc1bef143add-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.588658 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.588634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c666h\" (UniqueName: \"kubernetes.io/projected/68f91e4a-d089-4ce5-badf-dc1bef143add-kube-api-access-c666h\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rbq9t\" (UID: \"68f91e4a-d089-4ce5-badf-dc1bef143add\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.737470 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.737365    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:12.882455 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:12.882424    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"]
Jul 29 07:25:12.885497 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:25:12.885473    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f91e4a_d089_4ce5_badf_dc1bef143add.slice/crio-7beacfa0eae01e17527690e64c5fef9bee859024ebe1c8c56a5766cbfb52bb69 WatchSource:0}: Error finding container 7beacfa0eae01e17527690e64c5fef9bee859024ebe1c8c56a5766cbfb52bb69: Status 404 returned error can't find the container with id 7beacfa0eae01e17527690e64c5fef9bee859024ebe1c8c56a5766cbfb52bb69
Jul 29 07:25:13.315133 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:13.315095    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t" event={"ID":"68f91e4a-d089-4ce5-badf-dc1bef143add","Type":"ContainerStarted","Data":"19c59565c401675cd73c16bd2ed902ac4332740b03c58ab7e16cda48668fdb78"}
Jul 29 07:25:13.315366 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:13.315149    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t" event={"ID":"68f91e4a-d089-4ce5-badf-dc1bef143add","Type":"ContainerStarted","Data":"7beacfa0eae01e17527690e64c5fef9bee859024ebe1c8c56a5766cbfb52bb69"}
Jul 29 07:25:13.342931 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:13.342866    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t" podStartSLOduration=1.093700482 podStartE2EDuration="1.342849981s" totalImagesPullingTime="249.149499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:25:13.340466715 +0000 UTC m=+595.088532557" watchObservedRunningTime="2026-07-29 07:25:13.342849981 +0000 UTC m=+595.090915822"
Jul 29 07:25:15.416226 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:15.416192    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:25:17.673609 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.673569    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"]
Jul 29 07:25:17.676845 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.676828    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.680490 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.680472    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 07:25:17.718916 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.718869    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.719129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.718938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.719129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.718966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.719129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.718987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.719129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.719049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z94dt\" (UniqueName: \"kubernetes.io/projected/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-kube-api-access-z94dt\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.719129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.719063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"]
Jul 29 07:25:17.719129 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.719126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.819696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.819657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.819895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.819712    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.819895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.819744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.819895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.819775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.819895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.819806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z94dt\" (UniqueName: \"kubernetes.io/projected/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-kube-api-access-z94dt\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.819895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.819855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.820164 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.820147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.820217 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.820181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.820275 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.820222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.822098 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.822076    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.822353 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.822337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.842252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.842229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z94dt\" (UniqueName: \"kubernetes.io/projected/fedebd96-4830-4bfd-8e96-ba4bc891f7ef-kube-api-access-z94dt\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-zz58q\" (UID: \"fedebd96-4830-4bfd-8e96-ba4bc891f7ef\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:17.985981 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:17.985889    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:18.156330 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:18.156299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"]
Jul 29 07:25:18.157101 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:25:18.157079    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfedebd96_4830_4bfd_8e96_ba4bc891f7ef.slice/crio-5ac89b519f6bb3a9b95a027b94d4a01c42a68333da3e88a4601a092e982ae1e7 WatchSource:0}: Error finding container 5ac89b519f6bb3a9b95a027b94d4a01c42a68333da3e88a4601a092e982ae1e7: Status 404 returned error can't find the container with id 5ac89b519f6bb3a9b95a027b94d4a01c42a68333da3e88a4601a092e982ae1e7
Jul 29 07:25:18.337325 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:18.337248    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q" event={"ID":"fedebd96-4830-4bfd-8e96-ba4bc891f7ef","Type":"ContainerStarted","Data":"5ac89b519f6bb3a9b95a027b94d4a01c42a68333da3e88a4601a092e982ae1e7"}
Jul 29 07:25:19.343413 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:19.343370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q" event={"ID":"fedebd96-4830-4bfd-8e96-ba4bc891f7ef","Type":"ContainerStarted","Data":"1242bc13f29f259cb7b54442a6af13d8760dc4ab8df1a4a58990d105dd9aae29"}
Jul 29 07:25:19.402396 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:19.402340    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q" podStartSLOduration=2.160316432 podStartE2EDuration="2.402323297s" totalImagesPullingTime="242.006865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:25:19.400317464 +0000 UTC m=+601.148383305" watchObservedRunningTime="2026-07-29 07:25:19.402323297 +0000 UTC m=+601.150389138"
Jul 29 07:25:22.737519 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:22.737480    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:22.737972 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:22.737560    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:22.750057 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:22.750030    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:23.373585 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:23.373558    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rbq9t"
Jul 29 07:25:27.986631 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:27.986595    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:27.989032 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:27.986643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:27.999214 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:27.999192    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:28.395045 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:28.395014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-zz58q"
Jul 29 07:25:28.902868 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:25:28.902837    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:26:46.095976 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:26:46.095936    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:26:55.983230 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:26:55.983189    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:27:05.703851 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:27:05.703820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:28:47.606683 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:28:47.606598    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:29:01.394202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:29:01.394165    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:29:41.989569 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:29:41.989536    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:29:51.383518 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:29:51.383477    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:30:02.393405 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:30:02.393371    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:30:10.193589 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:30:10.193551    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:30:23.525202 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:30:23.525162    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:30:33.291088 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:30:33.291057    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:30:45.185855 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:30:45.185820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:30:54.318979 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:30:54.318939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:31:16.794009 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:31:16.793977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:31:28.106165 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:31:28.106125    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:31:54.308843 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:31:54.308763    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:32:08.109968 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:32:08.109928    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:32:18.387991 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:32:18.387954    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:32:35.991941 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:32:35.991325    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:32:45.782407 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:32:45.782369    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:32:57.883304 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:32:57.883267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:35:04.907893 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:35:04.907856    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:35:15.810124 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:35:15.810091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:35:30.693986 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:35:30.693953    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:35:40.083870 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:35:40.083832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:35:53.999331 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:35:53.999294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:36:02.623730 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:36:02.623692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:36:12.741483 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:36:12.741390    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:36:24.508361 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:36:24.508324    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:36:33.095120 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:36:33.095088    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:36:43.091489 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:36:43.091449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:36:56.101403 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:36:56.101358    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:37:04.682492 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:04.682456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:37:25.794297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:25.794259    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:37:33.851649 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:33.851600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:37:47.216656 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.216574    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"]
Jul 29 07:37:47.219949 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.219929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.223252 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.223232    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-be281498-kserve-self-signed-certs\""
Jul 29 07:37:47.270829 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.270801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-tmp-dir\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.270829 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.270829    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5356b231-ab9b-4210-8083-2c1ca80271f9-tls-certs\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.271016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.270861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-model-cache\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.271016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.270984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-dshm\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.271016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.271010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gctdk\" (UniqueName: \"kubernetes.io/projected/5356b231-ab9b-4210-8083-2c1ca80271f9-kube-api-access-gctdk\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.271117 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.271028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-home\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.321201 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.321171    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"]
Jul 29 07:37:47.371892 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.371853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-tmp-dir\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372087 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.371895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5356b231-ab9b-4210-8083-2c1ca80271f9-tls-certs\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372087 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.371979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-model-cache\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372087 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.372057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-dshm\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372087 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.372080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gctdk\" (UniqueName: \"kubernetes.io/projected/5356b231-ab9b-4210-8083-2c1ca80271f9-kube-api-access-gctdk\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.372107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-home\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372356 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.372323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-tmp-dir\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372356 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.372336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-model-cache\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.372480 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.372458    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-home\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.374115 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.374090    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-dshm\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.374453 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.374435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5356b231-ab9b-4210-8083-2c1ca80271f9-tls-certs\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.412244 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.412214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gctdk\" (UniqueName: \"kubernetes.io/projected/5356b231-ab9b-4210-8083-2c1ca80271f9-kube-api-access-gctdk\") pod \"e2e-same-id-a-be281498-kserve-57c7758475-66bvz\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") " pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.528633 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.528553    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:47.683974 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:47.683946    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"]
Jul 29 07:37:47.685006 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:37:47.684979    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5356b231_ab9b_4210_8083_2c1ca80271f9.slice/crio-b2b6e0fcc24608f1a294b0bb50815060cf19c1114b068162b74975d4f37eb213 WatchSource:0}: Error finding container b2b6e0fcc24608f1a294b0bb50815060cf19c1114b068162b74975d4f37eb213: Status 404 returned error can't find the container with id b2b6e0fcc24608f1a294b0bb50815060cf19c1114b068162b74975d4f37eb213
Jul 29 07:37:48.594463 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:48.594421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz" event={"ID":"5356b231-ab9b-4210-8083-2c1ca80271f9","Type":"ContainerStarted","Data":"f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a"}
Jul 29 07:37:48.594463 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:48.594469    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz" event={"ID":"5356b231-ab9b-4210-8083-2c1ca80271f9","Type":"ContainerStarted","Data":"b2b6e0fcc24608f1a294b0bb50815060cf19c1114b068162b74975d4f37eb213"}
Jul 29 07:37:48.629822 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:48.629769    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz" podStartSLOduration=1.629753255 podStartE2EDuration="1.629753255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:37:48.628618135 +0000 UTC m=+1350.376683987" watchObservedRunningTime="2026-07-29 07:37:48.629753255 +0000 UTC m=+1350.377819097"
Jul 29 07:37:50.434382 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:50.434349    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:37:57.528727 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:57.528688    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:57.529203 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:57.528746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:57.541130 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:57.541102    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:37:57.648139 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:37:57.648108    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:38:06.297640 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:06.297601    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:38:24.735631 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:24.735591    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"]
Jul 29 07:38:28.691778 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:28.691740    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:38:39.754355 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:39.754288    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz" podUID="5356b231-ab9b-4210-8083-2c1ca80271f9" containerName="main" containerID="cri-o://f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a" gracePeriod=45
Jul 29 07:38:39.987748 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:39.987723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:38:40.047343 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047266    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5356b231-ab9b-4210-8083-2c1ca80271f9-tls-certs\") pod \"5356b231-ab9b-4210-8083-2c1ca80271f9\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") "
Jul 29 07:38:40.047343 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047301    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-dshm\") pod \"5356b231-ab9b-4210-8083-2c1ca80271f9\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") "
Jul 29 07:38:40.047343 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047337    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-tmp-dir\") pod \"5356b231-ab9b-4210-8083-2c1ca80271f9\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") "
Jul 29 07:38:40.047569 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047409    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-model-cache\") pod \"5356b231-ab9b-4210-8083-2c1ca80271f9\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") "
Jul 29 07:38:40.047569 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gctdk\" (UniqueName: \"kubernetes.io/projected/5356b231-ab9b-4210-8083-2c1ca80271f9-kube-api-access-gctdk\") pod \"5356b231-ab9b-4210-8083-2c1ca80271f9\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") "
Jul 29 07:38:40.047569 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047489    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-home\") pod \"5356b231-ab9b-4210-8083-2c1ca80271f9\" (UID: \"5356b231-ab9b-4210-8083-2c1ca80271f9\") "
Jul 29 07:38:40.047726 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047648    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5356b231-ab9b-4210-8083-2c1ca80271f9" (UID: "5356b231-ab9b-4210-8083-2c1ca80271f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:38:40.047786 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047735    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "5356b231-ab9b-4210-8083-2c1ca80271f9" (UID: "5356b231-ab9b-4210-8083-2c1ca80271f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:38:40.047844 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047761    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:38:40.047844 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.047816    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-home" (OuterVolumeSpecName: "home") pod "5356b231-ab9b-4210-8083-2c1ca80271f9" (UID: "5356b231-ab9b-4210-8083-2c1ca80271f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:38:40.049558 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.049526    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-dshm" (OuterVolumeSpecName: "dshm") pod "5356b231-ab9b-4210-8083-2c1ca80271f9" (UID: "5356b231-ab9b-4210-8083-2c1ca80271f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:38:40.049661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.049556    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5356b231-ab9b-4210-8083-2c1ca80271f9-kube-api-access-gctdk" (OuterVolumeSpecName: "kube-api-access-gctdk") pod "5356b231-ab9b-4210-8083-2c1ca80271f9" (UID: "5356b231-ab9b-4210-8083-2c1ca80271f9"). InnerVolumeSpecName "kube-api-access-gctdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:38:40.049661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.049579    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5356b231-ab9b-4210-8083-2c1ca80271f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5356b231-ab9b-4210-8083-2c1ca80271f9" (UID: "5356b231-ab9b-4210-8083-2c1ca80271f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:38:40.148232 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.148199    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:38:40.148232 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.148227    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gctdk\" (UniqueName: \"kubernetes.io/projected/5356b231-ab9b-4210-8083-2c1ca80271f9-kube-api-access-gctdk\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:38:40.148232 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.148238    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:38:40.148506 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.148246    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5356b231-ab9b-4210-8083-2c1ca80271f9-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:38:40.148506 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.148255    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5356b231-ab9b-4210-8083-2c1ca80271f9-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:38:40.825546 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.825506    2577 generic.go:356] "Generic (PLEG): container finished" podID="5356b231-ab9b-4210-8083-2c1ca80271f9" containerID="f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a" exitCode=0
Jul 29 07:38:40.825976 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.825552    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"
Jul 29 07:38:40.825976 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.825555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz" event={"ID":"5356b231-ab9b-4210-8083-2c1ca80271f9","Type":"ContainerDied","Data":"f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a"}
Jul 29 07:38:40.825976 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.825593    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz" event={"ID":"5356b231-ab9b-4210-8083-2c1ca80271f9","Type":"ContainerDied","Data":"b2b6e0fcc24608f1a294b0bb50815060cf19c1114b068162b74975d4f37eb213"}
Jul 29 07:38:40.825976 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.825615    2577 scope.go:117] "RemoveContainer" containerID="f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a"
Jul 29 07:38:40.836156 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.836134    2577 scope.go:117] "RemoveContainer" containerID="f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a"
Jul 29 07:38:40.836450 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:38:40.836426    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a\": container with ID starting with f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a not found: ID does not exist" containerID="f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a"
Jul 29 07:38:40.836509 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.836459    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a"} err="failed to get container status \"f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a\": rpc error: code = NotFound desc = could not find container \"f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a\": container with ID starting with f52169e594b58e56839c861926f82fd49cb48711ea473c92c595a890410b5a4a not found: ID does not exist"
Jul 29 07:38:40.845485 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.845459    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"]
Jul 29 07:38:40.850093 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:40.850072    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-be281498-kserve-57c7758475-66bvz"]
Jul 29 07:38:42.298509 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:42.298478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:38:42.789576 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:42.789545    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5356b231-ab9b-4210-8083-2c1ca80271f9" path="/var/lib/kubelet/pods/5356b231-ab9b-4210-8083-2c1ca80271f9/volumes"
Jul 29 07:38:52.007393 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:38:52.007357    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:39:08.492661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:39:08.492624    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:39:19.323062 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:39:19.323022    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:39:29.114535 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:39:29.114497    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:39:44.903508 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:39:44.903470    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:39:54.724051 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:39:54.724012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:40:08.998130 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:40:08.998098    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:40:25.603079 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:40:25.603037    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:40:38.109657 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:40:38.109621    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:40:49.790663 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:40:49.790585    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:40:58.793346 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:40:58.793314    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:41:11.919306 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:41:11.919266    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:41:25.194153 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:41:25.194114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:41:35.091734 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:41:35.091700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:41:47.784619 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:41:47.784580    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:42:03.189991 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:42:03.189955    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:42:21.061164 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:42:21.061123    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:42:48.651933 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:42:48.651881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:43:03.854791 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:43:03.854753    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:43:26.859182 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:43:26.859137    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:43:31.397928 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:43:31.397878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:44:01.794321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:44:01.794283    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:44:16.402696 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:44:16.402658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:44:36.389362 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:44:36.389325    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:46:02.280055 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:46:02.280016    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:46:11.999725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:46:11.999687    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:46:24.124365 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:46:24.124322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:47:09.464592 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:47:09.464553    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:47:49.815151 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:47:49.815119    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:48:04.067527 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.067489    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"]
Jul 29 07:48:04.067931 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.067807    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5356b231-ab9b-4210-8083-2c1ca80271f9" containerName="main"
Jul 29 07:48:04.067931 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.067817    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5356b231-ab9b-4210-8083-2c1ca80271f9" containerName="main"
Jul 29 07:48:04.067931 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.067888    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5356b231-ab9b-4210-8083-2c1ca80271f9" containerName="main"
Jul 29 07:48:04.070718 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.070700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.102106 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.102081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-dshm\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.102231 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.102113    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np9s9\" (UniqueName: \"kubernetes.io/projected/febb3beb-8bad-4fa6-8937-4a4f30b02739-kube-api-access-np9s9\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.102231 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.102148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febb3beb-8bad-4fa6-8937-4a4f30b02739-tls-certs\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.102231 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.102211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-home\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.102364 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.102271    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-tmp-dir\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.102364 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.102303    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-model-cache\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.106807 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.106784    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-412a3f1a\"/\"kube-root-ca.crt\""
Jul 29 07:48:04.106895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.106836    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-412a3f1a\"/\"default-dockercfg-fw98h\""
Jul 29 07:48:04.106895 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.106867    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-412a3f1a\"/\"e2e-lifecycle-model-412a3f1a-kserve-self-signed-certs\""
Jul 29 07:48:04.106993 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.106787    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-412a3f1a\"/\"openshift-service-ca.crt\""
Jul 29 07:48:04.149120 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.149056    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"]
Jul 29 07:48:04.203364 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203322    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-home\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203559 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-tmp-dir\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203559 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-model-cache\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203559 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-dshm\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203707 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np9s9\" (UniqueName: \"kubernetes.io/projected/febb3beb-8bad-4fa6-8937-4a4f30b02739-kube-api-access-np9s9\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203707 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febb3beb-8bad-4fa6-8937-4a4f30b02739-tls-certs\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203808 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203786    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-home\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203863 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-model-cache\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.203863 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.203839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-tmp-dir\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.205792 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.205771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-dshm\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.206008 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.205986    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febb3beb-8bad-4fa6-8937-4a4f30b02739-tls-certs\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.236839 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.236808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np9s9\" (UniqueName: \"kubernetes.io/projected/febb3beb-8bad-4fa6-8937-4a4f30b02739-kube-api-access-np9s9\") pod \"e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") " pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.379887 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.379795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:04.532850 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:04.532818    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"]
Jul 29 07:48:04.536103 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:48:04.536070    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfebb3beb_8bad_4fa6_8937_4a4f30b02739.slice/crio-cc282afd7ade257a6b4e6a07edeebaffa0a80036ca0181e245d9549e82d786b7 WatchSource:0}: Error finding container cc282afd7ade257a6b4e6a07edeebaffa0a80036ca0181e245d9549e82d786b7: Status 404 returned error can't find the container with id cc282afd7ade257a6b4e6a07edeebaffa0a80036ca0181e245d9549e82d786b7
Jul 29 07:48:05.274934 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:05.274878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn" event={"ID":"febb3beb-8bad-4fa6-8937-4a4f30b02739","Type":"ContainerStarted","Data":"7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7"}
Jul 29 07:48:05.275337 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:05.274947    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn" event={"ID":"febb3beb-8bad-4fa6-8937-4a4f30b02739","Type":"ContainerStarted","Data":"cc282afd7ade257a6b4e6a07edeebaffa0a80036ca0181e245d9549e82d786b7"}
Jul 29 07:48:05.308110 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:05.308060    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn" podStartSLOduration=1.308046531 podStartE2EDuration="1.308046531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:48:05.30788866 +0000 UTC m=+1967.055954501" watchObservedRunningTime="2026-07-29 07:48:05.308046531 +0000 UTC m=+1967.056112372"
Jul 29 07:48:07.737584 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:07.737553    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:48:14.380813 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:14.380723    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:14.380813 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:14.380801    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:14.393255 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:14.393235    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:15.330752 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:15.330718    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:48:26.402930 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:26.402878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:48:44.280983 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:44.280948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:48:54.441193 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:54.441157    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:48:55.492583 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:48:55.492548    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"]
Jul 29 07:49:10.511450 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.511391    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn" podUID="febb3beb-8bad-4fa6-8937-4a4f30b02739" containerName="main" containerID="cri-o://7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7" gracePeriod=45
Jul 29 07:49:10.755411 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.755387    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:49:10.890778 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.890748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-home\") pod \"febb3beb-8bad-4fa6-8937-4a4f30b02739\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") "
Jul 29 07:49:10.890979 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.890805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-dshm\") pod \"febb3beb-8bad-4fa6-8937-4a4f30b02739\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") "
Jul 29 07:49:10.890979 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.890852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-tmp-dir\") pod \"febb3beb-8bad-4fa6-8937-4a4f30b02739\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") "
Jul 29 07:49:10.890979 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.890871    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np9s9\" (UniqueName: \"kubernetes.io/projected/febb3beb-8bad-4fa6-8937-4a4f30b02739-kube-api-access-np9s9\") pod \"febb3beb-8bad-4fa6-8937-4a4f30b02739\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") "
Jul 29 07:49:10.890979 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.890893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febb3beb-8bad-4fa6-8937-4a4f30b02739-tls-certs\") pod \"febb3beb-8bad-4fa6-8937-4a4f30b02739\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") "
Jul 29 07:49:10.891193 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.890978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-model-cache\") pod \"febb3beb-8bad-4fa6-8937-4a4f30b02739\" (UID: \"febb3beb-8bad-4fa6-8937-4a4f30b02739\") "
Jul 29 07:49:10.891193 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.891111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-home" (OuterVolumeSpecName: "home") pod "febb3beb-8bad-4fa6-8937-4a4f30b02739" (UID: "febb3beb-8bad-4fa6-8937-4a4f30b02739"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:49:10.891193 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.891130    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "febb3beb-8bad-4fa6-8937-4a4f30b02739" (UID: "febb3beb-8bad-4fa6-8937-4a4f30b02739"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:49:10.891738 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.891689    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-model-cache" (OuterVolumeSpecName: "model-cache") pod "febb3beb-8bad-4fa6-8937-4a4f30b02739" (UID: "febb3beb-8bad-4fa6-8937-4a4f30b02739"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:49:10.891844 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.891820    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:49:10.891915 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.891851    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:49:10.895951 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.894077    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/febb3beb-8bad-4fa6-8937-4a4f30b02739-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "febb3beb-8bad-4fa6-8937-4a4f30b02739" (UID: "febb3beb-8bad-4fa6-8937-4a4f30b02739"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:49:10.898300 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.898267    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-dshm" (OuterVolumeSpecName: "dshm") pod "febb3beb-8bad-4fa6-8937-4a4f30b02739" (UID: "febb3beb-8bad-4fa6-8937-4a4f30b02739"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:49:10.899111 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.899089    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/febb3beb-8bad-4fa6-8937-4a4f30b02739-kube-api-access-np9s9" (OuterVolumeSpecName: "kube-api-access-np9s9") pod "febb3beb-8bad-4fa6-8937-4a4f30b02739" (UID: "febb3beb-8bad-4fa6-8937-4a4f30b02739"). InnerVolumeSpecName "kube-api-access-np9s9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:49:10.992286 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.992254    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np9s9\" (UniqueName: \"kubernetes.io/projected/febb3beb-8bad-4fa6-8937-4a4f30b02739-kube-api-access-np9s9\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:49:10.992286 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.992281    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/febb3beb-8bad-4fa6-8937-4a4f30b02739-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:49:10.992286 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.992292    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:49:10.992479 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:10.992300    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/febb3beb-8bad-4fa6-8937-4a4f30b02739-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:49:11.563800 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.563758    2577 generic.go:356] "Generic (PLEG): container finished" podID="febb3beb-8bad-4fa6-8937-4a4f30b02739" containerID="7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7" exitCode=0
Jul 29 07:49:11.564297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.563824    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"
Jul 29 07:49:11.564297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.563844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn" event={"ID":"febb3beb-8bad-4fa6-8937-4a4f30b02739","Type":"ContainerDied","Data":"7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7"}
Jul 29 07:49:11.564297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.563931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn" event={"ID":"febb3beb-8bad-4fa6-8937-4a4f30b02739","Type":"ContainerDied","Data":"cc282afd7ade257a6b4e6a07edeebaffa0a80036ca0181e245d9549e82d786b7"}
Jul 29 07:49:11.564297 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.563960    2577 scope.go:117] "RemoveContainer" containerID="7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7"
Jul 29 07:49:11.572874 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.572856    2577 scope.go:117] "RemoveContainer" containerID="7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7"
Jul 29 07:49:11.573144 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:49:11.573125    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7\": container with ID starting with 7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7 not found: ID does not exist" containerID="7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7"
Jul 29 07:49:11.573211 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.573157    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7"} err="failed to get container status \"7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7\": rpc error: code = NotFound desc = could not find container \"7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7\": container with ID starting with 7227e0bdd1bfa575f8b6a2ebb56852935a5272426f33f47c6f322143b073abb7 not found: ID does not exist"
Jul 29 07:49:11.601800 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.601765    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"]
Jul 29 07:49:11.619672 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:11.619642    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-412a3f1a/e2e-lifecycle-model-412a3f1a-kserve-65d85bb495-jbhgn"]
Jul 29 07:49:12.790172 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:12.790134    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="febb3beb-8bad-4fa6-8937-4a4f30b02739" path="/var/lib/kubelet/pods/febb3beb-8bad-4fa6-8937-4a4f30b02739/volumes"
Jul 29 07:49:45.705360 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:45.705278    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:49:56.792887 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:49:56.792849    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:50:54.808376 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:50:54.808338    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:51:25.392960 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:51:25.392860    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:52:03.139749 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.139710    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"]
Jul 29 07:52:03.140232 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.140071    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="febb3beb-8bad-4fa6-8937-4a4f30b02739" containerName="main"
Jul 29 07:52:03.140232 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.140082    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="febb3beb-8bad-4fa6-8937-4a4f30b02739" containerName="main"
Jul 29 07:52:03.140232 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.140141    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="febb3beb-8bad-4fa6-8937-4a4f30b02739" containerName="main"
Jul 29 07:52:03.143018 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.142995    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.146661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.146640    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7d6491\"/\"default-dockercfg-vxw49\""
Jul 29 07:52:03.147974 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.147954    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-7d6491\"/\"test-model-7d6491-kserve-self-signed-certs\""
Jul 29 07:52:03.148076 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.148031    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7d6491\"/\"kube-root-ca.crt\""
Jul 29 07:52:03.148076 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.148058    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-7d6491\"/\"openshift-service-ca.crt\""
Jul 29 07:52:03.160071 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.160049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"]
Jul 29 07:52:03.187361 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.187332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-dshm\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.187469 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.187364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tmp-dir\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.187469 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.187395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctjsf\" (UniqueName: \"kubernetes.io/projected/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-kube-api-access-ctjsf\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.187469 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.187454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-home\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.187595 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.187483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tls-certs\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.187595 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.187520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-model-cache\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.287981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-model-cache\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288223 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-dshm\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288223 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tmp-dir\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288223 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctjsf\" (UniqueName: \"kubernetes.io/projected/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-kube-api-access-ctjsf\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288223 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-home\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288223 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288222    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tls-certs\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288481 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-model-cache\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288481 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tmp-dir\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.288587 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.288560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-home\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.290377 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.290346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-dshm\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.290650 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.290633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tls-certs\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.303603 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.303580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctjsf\" (UniqueName: \"kubernetes.io/projected/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-kube-api-access-ctjsf\") pod \"test-model-7d6491-kserve-65957949c5-m2tqx\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") " pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.453276 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.453188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:03.597967 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:03.597938    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"]
Jul 29 07:52:03.600614 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:52:03.600575    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbac5daaf_87b9_42f0_af08_5b7ace4b5bc4.slice/crio-907f9a28e407b102ef66fb0898087351bbb0480daae506700684f8ec7e78fbf1 WatchSource:0}: Error finding container 907f9a28e407b102ef66fb0898087351bbb0480daae506700684f8ec7e78fbf1: Status 404 returned error can't find the container with id 907f9a28e407b102ef66fb0898087351bbb0480daae506700684f8ec7e78fbf1
Jul 29 07:52:04.318264 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:04.318229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx" event={"ID":"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4","Type":"ContainerStarted","Data":"007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1"}
Jul 29 07:52:04.318659 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:04.318271    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx" event={"ID":"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4","Type":"ContainerStarted","Data":"907f9a28e407b102ef66fb0898087351bbb0480daae506700684f8ec7e78fbf1"}
Jul 29 07:52:04.342124 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:04.342076    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx" podStartSLOduration=1.342061239 podStartE2EDuration="1.342061239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:52:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:52:04.340864684 +0000 UTC m=+2206.088930526" watchObservedRunningTime="2026-07-29 07:52:04.342061239 +0000 UTC m=+2206.090127078"
Jul 29 07:52:06.634616 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:06.634575    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:52:13.453993 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:13.453958    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:13.454376 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:13.454009    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:13.466596 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:13.466567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:14.374434 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:14.374406    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:52:21.124731 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:21.124693    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:52:43.703920 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.703819    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"]
Jul 29 07:52:43.707278 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.707261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.710373 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.710352    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d6451b\"/\"kube-root-ca.crt\""
Jul 29 07:52:43.711762 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.711738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d6451b\"/\"test-model-d6451b-kserve-self-signed-certs\""
Jul 29 07:52:43.711890 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.711737    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d6451b\"/\"openshift-service-ca.crt\""
Jul 29 07:52:43.712039 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.712024    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d6451b\"/\"default-dockercfg-52tvr\""
Jul 29 07:52:43.721360 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.721337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"]
Jul 29 07:52:43.745571 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.745539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-dshm\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.745571 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.745571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6567b803-912f-4d6c-b44e-bac32547c90f-tls-certs\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.745725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.745594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk6f4\" (UniqueName: \"kubernetes.io/projected/6567b803-912f-4d6c-b44e-bac32547c90f-kube-api-access-pk6f4\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.745725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.745617    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-model-cache\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.745725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.745669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-home\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.745725 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.745692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-tmp-dir\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847150 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847102    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-dshm\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847150 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6567b803-912f-4d6c-b44e-bac32547c90f-tls-certs\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847360 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk6f4\" (UniqueName: \"kubernetes.io/projected/6567b803-912f-4d6c-b44e-bac32547c90f-kube-api-access-pk6f4\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847360 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-model-cache\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847430 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847399    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-home\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847471 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-tmp-dir\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847681 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-model-cache\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847806 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847706    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-home\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.847806 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.847789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-tmp-dir\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.849461 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.849443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-dshm\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.849616 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.849600    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6567b803-912f-4d6c-b44e-bac32547c90f-tls-certs\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:43.859356 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:43.859331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk6f4\" (UniqueName: \"kubernetes.io/projected/6567b803-912f-4d6c-b44e-bac32547c90f-kube-api-access-pk6f4\") pod \"test-model-d6451b-kserve-59b457dc65-sntnl\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") " pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:44.016839 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:44.016755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:44.147411 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:44.147381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"]
Jul 29 07:52:44.150559 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:52:44.150530    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6567b803_912f_4d6c_b44e_bac32547c90f.slice/crio-f135d8a31026dbef78201388b08eb145fa9c7fcae05bd8ba659d603e0d04532d WatchSource:0}: Error finding container f135d8a31026dbef78201388b08eb145fa9c7fcae05bd8ba659d603e0d04532d: Status 404 returned error can't find the container with id f135d8a31026dbef78201388b08eb145fa9c7fcae05bd8ba659d603e0d04532d
Jul 29 07:52:44.496080 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:44.496043    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl" event={"ID":"6567b803-912f-4d6c-b44e-bac32547c90f","Type":"ContainerStarted","Data":"ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99"}
Jul 29 07:52:44.496080 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:44.496087    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl" event={"ID":"6567b803-912f-4d6c-b44e-bac32547c90f","Type":"ContainerStarted","Data":"f135d8a31026dbef78201388b08eb145fa9c7fcae05bd8ba659d603e0d04532d"}
Jul 29 07:52:44.521743 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:44.521694    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl" podStartSLOduration=1.5216794089999999 podStartE2EDuration="1.521679409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:52:44.521583636 +0000 UTC m=+2246.269649478" watchObservedRunningTime="2026-07-29 07:52:44.521679409 +0000 UTC m=+2246.269745251"
Jul 29 07:52:49.632918 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:49.632863    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:52:54.017158 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:54.017106    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:54.017158 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:54.017170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:54.030274 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:54.030249    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:52:54.559158 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:52:54.559130    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:53:23.063083 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:23.063047    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"]
Jul 29 07:53:24.060659 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:24.060625    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"]
Jul 29 07:53:31.198013 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:31.197969    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:53:38.081686 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.081622    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx" podUID="bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" containerName="main" containerID="cri-o://007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1" gracePeriod=45
Jul 29 07:53:38.325049 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.325024    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:53:38.455433 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455408    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-model-cache\") pod \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") "
Jul 29 07:53:38.455608 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455445    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tls-certs\") pod \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") "
Jul 29 07:53:38.455608 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455465    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tmp-dir\") pod \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") "
Jul 29 07:53:38.455608 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455485    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctjsf\" (UniqueName: \"kubernetes.io/projected/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-kube-api-access-ctjsf\") pod \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") "
Jul 29 07:53:38.455608 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-dshm\") pod \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") "
Jul 29 07:53:38.455608 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455598    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-home\") pod \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\" (UID: \"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4\") "
Jul 29 07:53:38.455879 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455708    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-model-cache" (OuterVolumeSpecName: "model-cache") pod "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" (UID: "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:38.455879 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455737    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" (UID: "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:38.456005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455934    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:38.456005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455939    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-home" (OuterVolumeSpecName: "home") pod "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" (UID: "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:38.456005 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.455954    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:38.457729 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.457699    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-kube-api-access-ctjsf" (OuterVolumeSpecName: "kube-api-access-ctjsf") pod "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" (UID: "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4"). InnerVolumeSpecName "kube-api-access-ctjsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:53:38.457729 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.457712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-dshm" (OuterVolumeSpecName: "dshm") pod "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" (UID: "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:38.457871 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.457728    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" (UID: "bac5daaf-87b9-42f0-af08-5b7ace4b5bc4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:53:38.556605 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.556579    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:38.556605 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.556602    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctjsf\" (UniqueName: \"kubernetes.io/projected/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-kube-api-access-ctjsf\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:38.556754 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.556612    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:38.556754 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.556621    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:38.747805 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.747711    2577 generic.go:356] "Generic (PLEG): container finished" podID="bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" containerID="007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1" exitCode=0
Jul 29 07:53:38.747805 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.747757    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"
Jul 29 07:53:38.748016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.747753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx" event={"ID":"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4","Type":"ContainerDied","Data":"007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1"}
Jul 29 07:53:38.748016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.747865    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx" event={"ID":"bac5daaf-87b9-42f0-af08-5b7ace4b5bc4","Type":"ContainerDied","Data":"907f9a28e407b102ef66fb0898087351bbb0480daae506700684f8ec7e78fbf1"}
Jul 29 07:53:38.748016 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.747885    2577 scope.go:117] "RemoveContainer" containerID="007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1"
Jul 29 07:53:38.758705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.757842    2577 scope.go:117] "RemoveContainer" containerID="007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1"
Jul 29 07:53:38.758705 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:53:38.758273    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1\": container with ID starting with 007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1 not found: ID does not exist" containerID="007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1"
Jul 29 07:53:38.758705 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.758311    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1"} err="failed to get container status \"007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1\": rpc error: code = NotFound desc = could not find container \"007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1\": container with ID starting with 007d93d099cee0a0cf733381e7cccb797b7a2d08ea1bc35122c36f60ebf068b1 not found: ID does not exist"
Jul 29 07:53:38.774059 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.774030    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"]
Jul 29 07:53:38.777069 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.777045    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-7d6491/test-model-7d6491-kserve-65957949c5-m2tqx"]
Jul 29 07:53:38.790159 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:38.790133    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" path="/var/lib/kubelet/pods/bac5daaf-87b9-42f0-af08-5b7ace4b5bc4/volumes"
Jul 29 07:53:39.079144 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.079051    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl" podUID="6567b803-912f-4d6c-b44e-bac32547c90f" containerName="main" containerID="cri-o://ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99" gracePeriod=45
Jul 29 07:53:39.328600 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.328574    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:53:39.464662 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.464633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-tmp-dir\") pod \"6567b803-912f-4d6c-b44e-bac32547c90f\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") "
Jul 29 07:53:39.464929 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.464671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk6f4\" (UniqueName: \"kubernetes.io/projected/6567b803-912f-4d6c-b44e-bac32547c90f-kube-api-access-pk6f4\") pod \"6567b803-912f-4d6c-b44e-bac32547c90f\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") "
Jul 29 07:53:39.464929 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.464691    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-home\") pod \"6567b803-912f-4d6c-b44e-bac32547c90f\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") "
Jul 29 07:53:39.464929 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.464743    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-dshm\") pod \"6567b803-912f-4d6c-b44e-bac32547c90f\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") "
Jul 29 07:53:39.464929 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.464764    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6567b803-912f-4d6c-b44e-bac32547c90f-tls-certs\") pod \"6567b803-912f-4d6c-b44e-bac32547c90f\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") "
Jul 29 07:53:39.464929 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.464787    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-model-cache\") pod \"6567b803-912f-4d6c-b44e-bac32547c90f\" (UID: \"6567b803-912f-4d6c-b44e-bac32547c90f\") "
Jul 29 07:53:39.465197 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.465002    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6567b803-912f-4d6c-b44e-bac32547c90f" (UID: "6567b803-912f-4d6c-b44e-bac32547c90f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:39.465197 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.465116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-home" (OuterVolumeSpecName: "home") pod "6567b803-912f-4d6c-b44e-bac32547c90f" (UID: "6567b803-912f-4d6c-b44e-bac32547c90f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:39.465197 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.465146    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-model-cache" (OuterVolumeSpecName: "model-cache") pod "6567b803-912f-4d6c-b44e-bac32547c90f" (UID: "6567b803-912f-4d6c-b44e-bac32547c90f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:39.467050 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.467026    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-dshm" (OuterVolumeSpecName: "dshm") pod "6567b803-912f-4d6c-b44e-bac32547c90f" (UID: "6567b803-912f-4d6c-b44e-bac32547c90f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 07:53:39.467126 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.467058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6567b803-912f-4d6c-b44e-bac32547c90f-kube-api-access-pk6f4" (OuterVolumeSpecName: "kube-api-access-pk6f4") pod "6567b803-912f-4d6c-b44e-bac32547c90f" (UID: "6567b803-912f-4d6c-b44e-bac32547c90f"). InnerVolumeSpecName "kube-api-access-pk6f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 07:53:39.467203 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.467174    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6567b803-912f-4d6c-b44e-bac32547c90f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6567b803-912f-4d6c-b44e-bac32547c90f" (UID: "6567b803-912f-4d6c-b44e-bac32547c90f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 07:53:39.566352 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.566318    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:39.566352 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.566344    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6567b803-912f-4d6c-b44e-bac32547c90f-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:39.566352 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.566355    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:39.566611 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.566381    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:39.566611 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.566390    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk6f4\" (UniqueName: \"kubernetes.io/projected/6567b803-912f-4d6c-b44e-bac32547c90f-kube-api-access-pk6f4\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:39.566611 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.566398    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6567b803-912f-4d6c-b44e-bac32547c90f-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 07:53:39.754651 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.754564    2577 generic.go:356] "Generic (PLEG): container finished" podID="6567b803-912f-4d6c-b44e-bac32547c90f" containerID="ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99" exitCode=0
Jul 29 07:53:39.754651 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.754607    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"
Jul 29 07:53:39.754838 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.754608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl" event={"ID":"6567b803-912f-4d6c-b44e-bac32547c90f","Type":"ContainerDied","Data":"ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99"}
Jul 29 07:53:39.754838 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.754732    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl" event={"ID":"6567b803-912f-4d6c-b44e-bac32547c90f","Type":"ContainerDied","Data":"f135d8a31026dbef78201388b08eb145fa9c7fcae05bd8ba659d603e0d04532d"}
Jul 29 07:53:39.754838 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.754764    2577 scope.go:117] "RemoveContainer" containerID="ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99"
Jul 29 07:53:39.765486 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.765470    2577 scope.go:117] "RemoveContainer" containerID="ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99"
Jul 29 07:53:39.765791 ip-10-0-137-55 kubenswrapper[2577]: E0729 07:53:39.765768    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99\": container with ID starting with ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99 not found: ID does not exist" containerID="ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99"
Jul 29 07:53:39.765893 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.765799    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99"} err="failed to get container status \"ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99\": rpc error: code = NotFound desc = could not find container \"ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99\": container with ID starting with ff29a1ef3e63c039aca3fb688020e4abef065401927071c4f4c02897d5edcf99 not found: ID does not exist"
Jul 29 07:53:39.792646 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.792619    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"]
Jul 29 07:53:39.812421 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:39.812393    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d6451b/test-model-d6451b-kserve-59b457dc65-sntnl"]
Jul 29 07:53:40.790003 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:40.789961    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6567b803-912f-4d6c-b44e-bac32547c90f" path="/var/lib/kubelet/pods/6567b803-912f-4d6c-b44e-bac32547c90f/volumes"
Jul 29 07:53:40.808229 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:40.808196    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:53:57.295437 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:53:57.295401    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:54:14.089758 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:54:14.089678    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:54:40.092925 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:54:40.092881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:55:23.296281 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:55:23.296242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:55:48.683748 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:55:48.683665    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:56:00.378560 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:56:00.378524    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:56:39.697132 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:56:39.697081    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:57:13.479669 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.479589    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"]
Jul 29 07:57:13.480149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.479933    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" containerName="main"
Jul 29 07:57:13.480149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.479945    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" containerName="main"
Jul 29 07:57:13.480149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.479954    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6567b803-912f-4d6c-b44e-bac32547c90f" containerName="main"
Jul 29 07:57:13.480149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.479960    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6567b803-912f-4d6c-b44e-bac32547c90f" containerName="main"
Jul 29 07:57:13.480149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.480008    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6567b803-912f-4d6c-b44e-bac32547c90f" containerName="main"
Jul 29 07:57:13.480149 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.480019    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bac5daaf-87b9-42f0-af08-5b7ace4b5bc4" containerName="main"
Jul 29 07:57:13.483205 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.483185    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.486099 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.486075    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"openshift-service-ca.crt\""
Jul 29 07:57:13.486099 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.486093    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"kube-root-ca.crt\""
Jul 29 07:57:13.487301 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.487281    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"default-dockercfg-2fzqx\""
Jul 29 07:57:13.487392 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.487308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"auth-test-model-984886-kserve-self-signed-certs\""
Jul 29 07:57:13.493534 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.493506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"]
Jul 29 07:57:13.540536 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.540504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-dshm\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.540678 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.540540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47brq\" (UniqueName: \"kubernetes.io/projected/0119202e-ecdd-4b30-89fd-b6b874da413f-kube-api-access-47brq\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.540678 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.540616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-home\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.540801 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.540695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-model-cache\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.540801 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.540745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-tmp-dir\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.540801 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.540770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0119202e-ecdd-4b30-89fd-b6b874da413f-tls-certs\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641170 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641138    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-model-cache\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-tmp-dir\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641194    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0119202e-ecdd-4b30-89fd-b6b874da413f-tls-certs\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-dshm\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47brq\" (UniqueName: \"kubernetes.io/projected/0119202e-ecdd-4b30-89fd-b6b874da413f-kube-api-access-47brq\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641321 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-home\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641589 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641540    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-model-cache\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641645 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-tmp-dir\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.641691 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.641655    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-home\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.643549 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.643530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-dshm\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.643730 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.643712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0119202e-ecdd-4b30-89fd-b6b874da413f-tls-certs\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.650621 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.650596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47brq\" (UniqueName: \"kubernetes.io/projected/0119202e-ecdd-4b30-89fd-b6b874da413f-kube-api-access-47brq\") pod \"auth-test-model-984886-kserve-794949b779-qkb8m\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") " pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.792325 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.792228    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:13.922929 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:13.922804    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"]
Jul 29 07:57:13.925581 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:57:13.925549    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0119202e_ecdd_4b30_89fd_b6b874da413f.slice/crio-0e028985241c50092ef9397094247fa0cd9c0926d511411133bd7a0475386c93 WatchSource:0}: Error finding container 0e028985241c50092ef9397094247fa0cd9c0926d511411133bd7a0475386c93: Status 404 returned error can't find the container with id 0e028985241c50092ef9397094247fa0cd9c0926d511411133bd7a0475386c93
Jul 29 07:57:14.686287 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:14.686247    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m" event={"ID":"0119202e-ecdd-4b30-89fd-b6b874da413f","Type":"ContainerStarted","Data":"bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064"}
Jul 29 07:57:14.686649 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:14.686295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m" event={"ID":"0119202e-ecdd-4b30-89fd-b6b874da413f","Type":"ContainerStarted","Data":"0e028985241c50092ef9397094247fa0cd9c0926d511411133bd7a0475386c93"}
Jul 29 07:57:14.720846 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:14.720801    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m" podStartSLOduration=1.7207872370000001 podStartE2EDuration="1.720787237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:57:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:57:14.71947035 +0000 UTC m=+2516.467536191" watchObservedRunningTime="2026-07-29 07:57:14.720787237 +0000 UTC m=+2516.468853057"
Jul 29 07:57:17.576332 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:17.576292    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:57:23.792471 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:23.792426    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:23.793092 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:23.792499    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:23.804997 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:23.804971    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:24.740288 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:24.740256    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 07:57:54.418661 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.418621    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"]
Jul 29 07:57:54.422108 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.422089    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.427436 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.427409    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"openshift-service-ca.crt\""
Jul 29 07:57:54.429017 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.428989    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"kube-root-ca.crt\""
Jul 29 07:57:54.429017 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.428992    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"default-dockercfg-w5ttt\""
Jul 29 07:57:54.429179 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.429116    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"auth-test-model-71e038-kserve-self-signed-certs\""
Jul 29 07:57:54.436229 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.436201    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"]
Jul 29 07:57:54.491074 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.491044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-home\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.491323 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.491092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-md54q\" (UniqueName: \"kubernetes.io/projected/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-kube-api-access-md54q\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.491323 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.491121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tmp-dir\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.491323 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.491140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-model-cache\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.491323 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.491161    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-dshm\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.491323 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.491185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tls-certs\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.591660 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.591623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-home\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.591850 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.591684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-md54q\" (UniqueName: \"kubernetes.io/projected/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-kube-api-access-md54q\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.591850 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.591725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tmp-dir\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.591850 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.591752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-model-cache\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.592065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.591881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-dshm\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.592065 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.591951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tls-certs\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.592177 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.592082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-home\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.592177 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.592144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-model-cache\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.592255 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.592180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tmp-dir\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.594093 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.594072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-dshm\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.594248 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.594230    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tls-certs\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.602986 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.602960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-md54q\" (UniqueName: \"kubernetes.io/projected/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-kube-api-access-md54q\") pod \"auth-test-model-71e038-kserve-8498d9ffd7-m5gkk\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") " pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.731011 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.730927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:57:54.879879 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:54.879844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"]
Jul 29 07:57:54.884014 ip-10-0-137-55 kubenswrapper[2577]: W0729 07:57:54.883983    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd19fc25e_5544_44f9_a7ed_cbe4975c6f73.slice/crio-6e11c83140f1c4602d42a31d9f26ddc977f09f818a880755095080fde4b8f0f4 WatchSource:0}: Error finding container 6e11c83140f1c4602d42a31d9f26ddc977f09f818a880755095080fde4b8f0f4: Status 404 returned error can't find the container with id 6e11c83140f1c4602d42a31d9f26ddc977f09f818a880755095080fde4b8f0f4
Jul 29 07:57:55.862461 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:55.862426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk" event={"ID":"d19fc25e-5544-44f9-a7ed-cbe4975c6f73","Type":"ContainerStarted","Data":"64e2932dc617f613eac54de183dffd22b46f60464214e5a00f2168275b01f253"}
Jul 29 07:57:55.862461 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:55.862467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk" event={"ID":"d19fc25e-5544-44f9-a7ed-cbe4975c6f73","Type":"ContainerStarted","Data":"6e11c83140f1c4602d42a31d9f26ddc977f09f818a880755095080fde4b8f0f4"}
Jul 29 07:57:55.883333 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:55.883283    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk" podStartSLOduration=1.883265878 podStartE2EDuration="1.883265878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 07:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 07:57:55.883053806 +0000 UTC m=+2557.631119649" watchObservedRunningTime="2026-07-29 07:57:55.883265878 +0000 UTC m=+2557.631331719"
Jul 29 07:57:58.096816 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:57:58.096779    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:58:04.731270 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:58:04.731234    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:58:04.731798 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:58:04.731286    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:58:04.748790 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:58:04.748762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:58:04.910913 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:58:04.910872    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 07:58:23.121743 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:58:23.121700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:58:48.896922 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:58:48.896819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:59:07.694389 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:59:07.694354    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:59:35.784700 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:59:35.784663    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 07:59:54.096015 ip-10-0-137-55 kubenswrapper[2577]: I0729 07:59:54.095977    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:00:22.192517 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:00:22.192483    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:00:41.008105 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:00:41.008068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:01:01.783363 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:01:01.783321    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:01:21.408896 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:01:21.408859    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:01:43.911807 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:01:43.911727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:02:01.999471 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:01.999429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:02:28.499229 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:28.499184    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:02:34.124282 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:34.124247    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"]
Jul 29 08:02:35.021212 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:35.021175    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"]
Jul 29 08:02:45.587446 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.587408    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"]
Jul 29 08:02:45.590807 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.590788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.595725 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.595700    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"sub-test-model-984886-kserve-self-signed-certs\""
Jul 29 08:02:45.605036 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.605013    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"]
Jul 29 08:02:45.628822 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.628798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-home\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.628958 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.628832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-model-cache\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.628958 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.628851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-tmp-dir\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.628958 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.628873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq9sm\" (UniqueName: \"kubernetes.io/projected/07926ca1-55fa-4de5-8cbd-592a267225e6-kube-api-access-bq9sm\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.629065 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.628952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07926ca1-55fa-4de5-8cbd-592a267225e6-tls-certs\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.629065 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.628986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-dshm\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730017 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.729979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-home\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730017 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-model-cache\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730239 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-tmp-dir\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730239 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq9sm\" (UniqueName: \"kubernetes.io/projected/07926ca1-55fa-4de5-8cbd-592a267225e6-kube-api-access-bq9sm\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730239 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07926ca1-55fa-4de5-8cbd-592a267225e6-tls-certs\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730239 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-dshm\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730476 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-home\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730542 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-tmp-dir\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.730542 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.730519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-model-cache\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.732417 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.732386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-dshm\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.732696 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.732674    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07926ca1-55fa-4de5-8cbd-592a267225e6-tls-certs\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.739674 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.739650    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq9sm\" (UniqueName: \"kubernetes.io/projected/07926ca1-55fa-4de5-8cbd-592a267225e6-kube-api-access-bq9sm\") pod \"sub-test-model-984886-kserve-665c654c9c-ts7zv\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") " pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:45.889628 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.889597    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:02:45.899466 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:45.899440    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:46.045989 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:46.045965    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"]
Jul 29 08:02:46.147135 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:46.147048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv" event={"ID":"07926ca1-55fa-4de5-8cbd-592a267225e6","Type":"ContainerStarted","Data":"13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874"}
Jul 29 08:02:46.147135 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:46.147088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv" event={"ID":"07926ca1-55fa-4de5-8cbd-592a267225e6","Type":"ContainerStarted","Data":"93feb3c9f0d15549e2f0726c9496f193b7ccad4e825b1324577ec3136d27ff24"}
Jul 29 08:02:46.171989 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:46.171933    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv" podStartSLOduration=1.171916496 podStartE2EDuration="1.171916496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:02:46.171124196 +0000 UTC m=+2847.919190038" watchObservedRunningTime="2026-07-29 08:02:46.171916496 +0000 UTC m=+2847.919982332"
Jul 29 08:02:49.143316 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.143254    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m" podUID="0119202e-ecdd-4b30-89fd-b6b874da413f" containerName="main" containerID="cri-o://bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064" gracePeriod=45
Jul 29 08:02:49.398237 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.398180    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 08:02:49.463117 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463091    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47brq\" (UniqueName: \"kubernetes.io/projected/0119202e-ecdd-4b30-89fd-b6b874da413f-kube-api-access-47brq\") pod \"0119202e-ecdd-4b30-89fd-b6b874da413f\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") "
Jul 29 08:02:49.463249 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463124    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0119202e-ecdd-4b30-89fd-b6b874da413f-tls-certs\") pod \"0119202e-ecdd-4b30-89fd-b6b874da413f\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") "
Jul 29 08:02:49.463249 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463152    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-model-cache\") pod \"0119202e-ecdd-4b30-89fd-b6b874da413f\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") "
Jul 29 08:02:49.463365 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463342    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-tmp-dir\") pod \"0119202e-ecdd-4b30-89fd-b6b874da413f\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") "
Jul 29 08:02:49.463417 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0119202e-ecdd-4b30-89fd-b6b874da413f" (UID: "0119202e-ecdd-4b30-89fd-b6b874da413f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:49.463478 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463415    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-dshm\") pod \"0119202e-ecdd-4b30-89fd-b6b874da413f\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") "
Jul 29 08:02:49.463478 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463431    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-home\") pod \"0119202e-ecdd-4b30-89fd-b6b874da413f\" (UID: \"0119202e-ecdd-4b30-89fd-b6b874da413f\") "
Jul 29 08:02:49.463637 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463614    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0119202e-ecdd-4b30-89fd-b6b874da413f" (UID: "0119202e-ecdd-4b30-89fd-b6b874da413f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:49.463745 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463724    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-home" (OuterVolumeSpecName: "home") pod "0119202e-ecdd-4b30-89fd-b6b874da413f" (UID: "0119202e-ecdd-4b30-89fd-b6b874da413f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:49.463807 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463758    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:49.463807 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.463780    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:49.465424 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.465402    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0119202e-ecdd-4b30-89fd-b6b874da413f-kube-api-access-47brq" (OuterVolumeSpecName: "kube-api-access-47brq") pod "0119202e-ecdd-4b30-89fd-b6b874da413f" (UID: "0119202e-ecdd-4b30-89fd-b6b874da413f"). InnerVolumeSpecName "kube-api-access-47brq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:02:49.465663 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.465644    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0119202e-ecdd-4b30-89fd-b6b874da413f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0119202e-ecdd-4b30-89fd-b6b874da413f" (UID: "0119202e-ecdd-4b30-89fd-b6b874da413f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:02:49.465719 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.465650    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-dshm" (OuterVolumeSpecName: "dshm") pod "0119202e-ecdd-4b30-89fd-b6b874da413f" (UID: "0119202e-ecdd-4b30-89fd-b6b874da413f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:49.564454 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.564418    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:49.564454 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.564450    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0119202e-ecdd-4b30-89fd-b6b874da413f-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:49.564454 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.564462    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47brq\" (UniqueName: \"kubernetes.io/projected/0119202e-ecdd-4b30-89fd-b6b874da413f-kube-api-access-47brq\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:49.564698 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:49.564475    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0119202e-ecdd-4b30-89fd-b6b874da413f-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:50.039794 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.039734    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk" podUID="d19fc25e-5544-44f9-a7ed-cbe4975c6f73" containerName="main" containerID="cri-o://64e2932dc617f613eac54de183dffd22b46f60464214e5a00f2168275b01f253" gracePeriod=45
Jul 29 08:02:50.167008 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.166975    2577 generic.go:356] "Generic (PLEG): container finished" podID="0119202e-ecdd-4b30-89fd-b6b874da413f" containerID="bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064" exitCode=0
Jul 29 08:02:50.167413 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.167027    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"
Jul 29 08:02:50.167413 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.167056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m" event={"ID":"0119202e-ecdd-4b30-89fd-b6b874da413f","Type":"ContainerDied","Data":"bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064"}
Jul 29 08:02:50.167413 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.167096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m" event={"ID":"0119202e-ecdd-4b30-89fd-b6b874da413f","Type":"ContainerDied","Data":"0e028985241c50092ef9397094247fa0cd9c0926d511411133bd7a0475386c93"}
Jul 29 08:02:50.167413 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.167122    2577 scope.go:117] "RemoveContainer" containerID="bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064"
Jul 29 08:02:50.169188 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.169161    2577 generic.go:356] "Generic (PLEG): container finished" podID="d19fc25e-5544-44f9-a7ed-cbe4975c6f73" containerID="64e2932dc617f613eac54de183dffd22b46f60464214e5a00f2168275b01f253" exitCode=0
Jul 29 08:02:50.169319 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.169285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk" event={"ID":"d19fc25e-5544-44f9-a7ed-cbe4975c6f73","Type":"ContainerDied","Data":"64e2932dc617f613eac54de183dffd22b46f60464214e5a00f2168275b01f253"}
Jul 29 08:02:50.175926 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.175829    2577 scope.go:117] "RemoveContainer" containerID="bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064"
Jul 29 08:02:50.176179 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:02:50.176158    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064\": container with ID starting with bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064 not found: ID does not exist" containerID="bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064"
Jul 29 08:02:50.176234 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.176188    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064"} err="failed to get container status \"bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064\": rpc error: code = NotFound desc = could not find container \"bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064\": container with ID starting with bd073ef98dde0718e3c673b517aad51248a52a105436a945f412595ea262d064 not found: ID does not exist"
Jul 29 08:02:50.300579 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.300518    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 08:02:50.371736 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.371710    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-home\") pod \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") "
Jul 29 08:02:50.371871 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.371751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-dshm\") pod \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") "
Jul 29 08:02:50.371871 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.371817    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tmp-dir\") pod \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") "
Jul 29 08:02:50.371997 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.371874    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-model-cache\") pod \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") "
Jul 29 08:02:50.371997 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.371934    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-md54q\" (UniqueName: \"kubernetes.io/projected/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-kube-api-access-md54q\") pod \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") "
Jul 29 08:02:50.371997 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.371959    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tls-certs\") pod \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\" (UID: \"d19fc25e-5544-44f9-a7ed-cbe4975c6f73\") "
Jul 29 08:02:50.372156 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.372029    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-home" (OuterVolumeSpecName: "home") pod "d19fc25e-5544-44f9-a7ed-cbe4975c6f73" (UID: "d19fc25e-5544-44f9-a7ed-cbe4975c6f73"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:50.372156 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.372107    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d19fc25e-5544-44f9-a7ed-cbe4975c6f73" (UID: "d19fc25e-5544-44f9-a7ed-cbe4975c6f73"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:50.372269 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.372185    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-model-cache" (OuterVolumeSpecName: "model-cache") pod "d19fc25e-5544-44f9-a7ed-cbe4975c6f73" (UID: "d19fc25e-5544-44f9-a7ed-cbe4975c6f73"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:50.372322 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.372306    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:50.372374 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.372326    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:50.372374 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.372340    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:50.374050 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.374031    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-dshm" (OuterVolumeSpecName: "dshm") pod "d19fc25e-5544-44f9-a7ed-cbe4975c6f73" (UID: "d19fc25e-5544-44f9-a7ed-cbe4975c6f73"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:02:50.374168 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.374147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d19fc25e-5544-44f9-a7ed-cbe4975c6f73" (UID: "d19fc25e-5544-44f9-a7ed-cbe4975c6f73"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:02:50.374208 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.374169    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-kube-api-access-md54q" (OuterVolumeSpecName: "kube-api-access-md54q") pod "d19fc25e-5544-44f9-a7ed-cbe4975c6f73" (UID: "d19fc25e-5544-44f9-a7ed-cbe4975c6f73"). InnerVolumeSpecName "kube-api-access-md54q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:02:50.473290 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.473258    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-md54q\" (UniqueName: \"kubernetes.io/projected/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-kube-api-access-md54q\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:50.473290 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.473286    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:50.473290 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:50.473296    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d19fc25e-5544-44f9-a7ed-cbe4975c6f73-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:02:51.176090 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.176053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk" event={"ID":"d19fc25e-5544-44f9-a7ed-cbe4975c6f73","Type":"ContainerDied","Data":"6e11c83140f1c4602d42a31d9f26ddc977f09f818a880755095080fde4b8f0f4"}
Jul 29 08:02:51.176558 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.176105    2577 scope.go:117] "RemoveContainer" containerID="64e2932dc617f613eac54de183dffd22b46f60464214e5a00f2168275b01f253"
Jul 29 08:02:51.176558 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.176067    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"
Jul 29 08:02:51.525484 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.525363    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"]
Jul 29 08:02:51.528916 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.528863    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-984886/auth-test-model-984886-kserve-794949b779-qkb8m"]
Jul 29 08:02:51.939766 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.939729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"]
Jul 29 08:02:51.946365 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:51.946339    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/auth-test-model-71e038-kserve-8498d9ffd7-m5gkk"]
Jul 29 08:02:52.790343 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:52.790309    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0119202e-ecdd-4b30-89fd-b6b874da413f" path="/var/lib/kubelet/pods/0119202e-ecdd-4b30-89fd-b6b874da413f/volumes"
Jul 29 08:02:52.790730 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:52.790701    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d19fc25e-5544-44f9-a7ed-cbe4975c6f73" path="/var/lib/kubelet/pods/d19fc25e-5544-44f9-a7ed-cbe4975c6f73/volumes"
Jul 29 08:02:55.899860 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:55.899818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:55.900365 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:55.899881    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:55.912559 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:55.912531    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:56.209744 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:56.209652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:02:59.328647 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:02:59.328610    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:03:11.701711 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.701628    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"]
Jul 29 08:03:11.702126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.701955    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0119202e-ecdd-4b30-89fd-b6b874da413f" containerName="main"
Jul 29 08:03:11.702126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.701966    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0119202e-ecdd-4b30-89fd-b6b874da413f" containerName="main"
Jul 29 08:03:11.702126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.701977    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d19fc25e-5544-44f9-a7ed-cbe4975c6f73" containerName="main"
Jul 29 08:03:11.702126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.701982    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d19fc25e-5544-44f9-a7ed-cbe4975c6f73" containerName="main"
Jul 29 08:03:11.702126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.702035    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d19fc25e-5544-44f9-a7ed-cbe4975c6f73" containerName="main"
Jul 29 08:03:11.702126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.702044    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0119202e-ecdd-4b30-89fd-b6b874da413f" containerName="main"
Jul 29 08:03:11.704877 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.704859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.709206 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.709180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"sub-test-model-71e038-kserve-self-signed-certs\""
Jul 29 08:03:11.709348 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.709207    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"openshift-service-ca.crt\""
Jul 29 08:03:11.709348 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.709222    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"kube-root-ca.crt\""
Jul 29 08:03:11.709348 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.709180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"default-dockercfg-w5ttt\""
Jul 29 08:03:11.717193 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.717169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"]
Jul 29 08:03:11.856691 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.856653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-dshm\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.856878 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.856704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-model-cache\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.856878 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.856729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-home\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.856878 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.856814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gwtm\" (UniqueName: \"kubernetes.io/projected/e5790b2f-88cb-461c-99ba-d7f0800178e2-kube-api-access-8gwtm\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.856878 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.856851    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5790b2f-88cb-461c-99ba-d7f0800178e2-tls-certs\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.857118 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.856977    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-tmp-dir\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958052 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.957955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-model-cache\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958052 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.957998    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-home\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958052 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gwtm\" (UniqueName: \"kubernetes.io/projected/e5790b2f-88cb-461c-99ba-d7f0800178e2-kube-api-access-8gwtm\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958308 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5790b2f-88cb-461c-99ba-d7f0800178e2-tls-certs\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958371 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-tmp-dir\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958431 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-dshm\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958492 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-model-cache\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958618 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958599    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-home\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.958717 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.958700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-tmp-dir\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.960627 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.960604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-dshm\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.960790 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.960772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5790b2f-88cb-461c-99ba-d7f0800178e2-tls-certs\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:11.967120 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:11.967094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gwtm\" (UniqueName: \"kubernetes.io/projected/e5790b2f-88cb-461c-99ba-d7f0800178e2-kube-api-access-8gwtm\") pod \"sub-test-model-71e038-kserve-69f44b675-rd7qv\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") " pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:12.013792 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:12.013762    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:12.143835 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:12.143807    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"]
Jul 29 08:03:12.146696 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:03:12.146664    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5790b2f_88cb_461c_99ba_d7f0800178e2.slice/crio-7da7e265e0dba5a62b00b761804049cb1774edb7da846ba8531c9bbe58937fa7 WatchSource:0}: Error finding container 7da7e265e0dba5a62b00b761804049cb1774edb7da846ba8531c9bbe58937fa7: Status 404 returned error can't find the container with id 7da7e265e0dba5a62b00b761804049cb1774edb7da846ba8531c9bbe58937fa7
Jul 29 08:03:12.274078 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:12.274044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv" event={"ID":"e5790b2f-88cb-461c-99ba-d7f0800178e2","Type":"ContainerStarted","Data":"cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c"}
Jul 29 08:03:12.274233 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:12.274091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv" event={"ID":"e5790b2f-88cb-461c-99ba-d7f0800178e2","Type":"ContainerStarted","Data":"7da7e265e0dba5a62b00b761804049cb1774edb7da846ba8531c9bbe58937fa7"}
Jul 29 08:03:12.293095 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:12.293047    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv" podStartSLOduration=1.293030827 podStartE2EDuration="1.293030827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:03:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:03:12.292284609 +0000 UTC m=+2874.040350451" watchObservedRunningTime="2026-07-29 08:03:12.293030827 +0000 UTC m=+2874.041096668"
Jul 29 08:03:15.317624 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:15.317591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:03:22.014834 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:22.014792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:22.015292 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:22.014847    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:22.027404 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:22.027379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:22.327551 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:22.327459    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:03:54.642128 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:03:54.642088    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:04:25.693643 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:25.693607    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:04:30.871934 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:30.871884    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"]
Jul 29 08:04:32.006769 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:32.006735    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"]
Jul 29 08:04:43.913420 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:43.913335    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:04:45.890675 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:45.890627    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv" podUID="07926ca1-55fa-4de5-8cbd-592a267225e6" containerName="main" containerID="cri-o://13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874" gracePeriod=45
Jul 29 08:04:46.134018 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.133991    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:04:46.295417 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295333    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-tmp-dir\") pod \"07926ca1-55fa-4de5-8cbd-592a267225e6\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") "
Jul 29 08:04:46.295417 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295376    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-home\") pod \"07926ca1-55fa-4de5-8cbd-592a267225e6\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") "
Jul 29 08:04:46.295630 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295489    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07926ca1-55fa-4de5-8cbd-592a267225e6-tls-certs\") pod \"07926ca1-55fa-4de5-8cbd-592a267225e6\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") "
Jul 29 08:04:46.295630 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295522    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-model-cache\") pod \"07926ca1-55fa-4de5-8cbd-592a267225e6\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") "
Jul 29 08:04:46.295630 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295565    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq9sm\" (UniqueName: \"kubernetes.io/projected/07926ca1-55fa-4de5-8cbd-592a267225e6-kube-api-access-bq9sm\") pod \"07926ca1-55fa-4de5-8cbd-592a267225e6\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") "
Jul 29 08:04:46.295788 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295655    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-dshm\") pod \"07926ca1-55fa-4de5-8cbd-592a267225e6\" (UID: \"07926ca1-55fa-4de5-8cbd-592a267225e6\") "
Jul 29 08:04:46.295788 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295646    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07926ca1-55fa-4de5-8cbd-592a267225e6" (UID: "07926ca1-55fa-4de5-8cbd-592a267225e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:46.295788 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295699    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-home" (OuterVolumeSpecName: "home") pod "07926ca1-55fa-4de5-8cbd-592a267225e6" (UID: "07926ca1-55fa-4de5-8cbd-592a267225e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:46.295979 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.295830    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "07926ca1-55fa-4de5-8cbd-592a267225e6" (UID: "07926ca1-55fa-4de5-8cbd-592a267225e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:46.296064 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.296048    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:46.296116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.296071    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:46.296116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.296086    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:46.297627 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.297598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07926ca1-55fa-4de5-8cbd-592a267225e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07926ca1-55fa-4de5-8cbd-592a267225e6" (UID: "07926ca1-55fa-4de5-8cbd-592a267225e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:04:46.297745 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.297636    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07926ca1-55fa-4de5-8cbd-592a267225e6-kube-api-access-bq9sm" (OuterVolumeSpecName: "kube-api-access-bq9sm") pod "07926ca1-55fa-4de5-8cbd-592a267225e6" (UID: "07926ca1-55fa-4de5-8cbd-592a267225e6"). InnerVolumeSpecName "kube-api-access-bq9sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:04:46.297745 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.297684    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-dshm" (OuterVolumeSpecName: "dshm") pod "07926ca1-55fa-4de5-8cbd-592a267225e6" (UID: "07926ca1-55fa-4de5-8cbd-592a267225e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:46.396528 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.396497    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07926ca1-55fa-4de5-8cbd-592a267225e6-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:46.396528 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.396522    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq9sm\" (UniqueName: \"kubernetes.io/projected/07926ca1-55fa-4de5-8cbd-592a267225e6-kube-api-access-bq9sm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:46.396528 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.396531    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07926ca1-55fa-4de5-8cbd-592a267225e6-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:46.691119 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.691080    2577 generic.go:356] "Generic (PLEG): container finished" podID="07926ca1-55fa-4de5-8cbd-592a267225e6" containerID="13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874" exitCode=0
Jul 29 08:04:46.691292 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.691158    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"
Jul 29 08:04:46.691292 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.691163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv" event={"ID":"07926ca1-55fa-4de5-8cbd-592a267225e6","Type":"ContainerDied","Data":"13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874"}
Jul 29 08:04:46.691292 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.691203    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv" event={"ID":"07926ca1-55fa-4de5-8cbd-592a267225e6","Type":"ContainerDied","Data":"93feb3c9f0d15549e2f0726c9496f193b7ccad4e825b1324577ec3136d27ff24"}
Jul 29 08:04:46.691292 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.691224    2577 scope.go:117] "RemoveContainer" containerID="13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874"
Jul 29 08:04:46.699888 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.699869    2577 scope.go:117] "RemoveContainer" containerID="13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874"
Jul 29 08:04:46.700190 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:04:46.700168    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874\": container with ID starting with 13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874 not found: ID does not exist" containerID="13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874"
Jul 29 08:04:46.700286 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.700196    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874"} err="failed to get container status \"13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874\": rpc error: code = NotFound desc = could not find container \"13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874\": container with ID starting with 13400f620eabc565fd6ac42859312e128613b33a8a2ec2d4e0777c7d396e0874 not found: ID does not exist"
Jul 29 08:04:46.733813 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.733777    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"]
Jul 29 08:04:46.737756 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.737730    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-984886/sub-test-model-984886-kserve-665c654c9c-ts7zv"]
Jul 29 08:04:46.790538 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:46.790507    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07926ca1-55fa-4de5-8cbd-592a267225e6" path="/var/lib/kubelet/pods/07926ca1-55fa-4de5-8cbd-592a267225e6/volumes"
Jul 29 08:04:47.025248 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.025158    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv" podUID="e5790b2f-88cb-461c-99ba-d7f0800178e2" containerName="main" containerID="cri-o://cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c" gracePeriod=45
Jul 29 08:04:47.271932 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.271882    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:04:47.405228 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-home\") pod \"e5790b2f-88cb-461c-99ba-d7f0800178e2\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") "
Jul 29 08:04:47.405228 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405232    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-tmp-dir\") pod \"e5790b2f-88cb-461c-99ba-d7f0800178e2\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") "
Jul 29 08:04:47.405491 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405288    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gwtm\" (UniqueName: \"kubernetes.io/projected/e5790b2f-88cb-461c-99ba-d7f0800178e2-kube-api-access-8gwtm\") pod \"e5790b2f-88cb-461c-99ba-d7f0800178e2\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") "
Jul 29 08:04:47.405491 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405312    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5790b2f-88cb-461c-99ba-d7f0800178e2-tls-certs\") pod \"e5790b2f-88cb-461c-99ba-d7f0800178e2\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") "
Jul 29 08:04:47.405491 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405330    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-model-cache\") pod \"e5790b2f-88cb-461c-99ba-d7f0800178e2\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") "
Jul 29 08:04:47.405491 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-dshm\") pod \"e5790b2f-88cb-461c-99ba-d7f0800178e2\" (UID: \"e5790b2f-88cb-461c-99ba-d7f0800178e2\") "
Jul 29 08:04:47.405710 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405523    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-home" (OuterVolumeSpecName: "home") pod "e5790b2f-88cb-461c-99ba-d7f0800178e2" (UID: "e5790b2f-88cb-461c-99ba-d7f0800178e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:47.405710 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405634    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "e5790b2f-88cb-461c-99ba-d7f0800178e2" (UID: "e5790b2f-88cb-461c-99ba-d7f0800178e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:47.405710 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.405658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e5790b2f-88cb-461c-99ba-d7f0800178e2" (UID: "e5790b2f-88cb-461c-99ba-d7f0800178e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:47.407611 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.407583    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-dshm" (OuterVolumeSpecName: "dshm") pod "e5790b2f-88cb-461c-99ba-d7f0800178e2" (UID: "e5790b2f-88cb-461c-99ba-d7f0800178e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:04:47.407724 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.407645    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5790b2f-88cb-461c-99ba-d7f0800178e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e5790b2f-88cb-461c-99ba-d7f0800178e2" (UID: "e5790b2f-88cb-461c-99ba-d7f0800178e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:04:47.407724 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.407655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5790b2f-88cb-461c-99ba-d7f0800178e2-kube-api-access-8gwtm" (OuterVolumeSpecName: "kube-api-access-8gwtm") pod "e5790b2f-88cb-461c-99ba-d7f0800178e2" (UID: "e5790b2f-88cb-461c-99ba-d7f0800178e2"). InnerVolumeSpecName "kube-api-access-8gwtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:04:47.506699 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.506661    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gwtm\" (UniqueName: \"kubernetes.io/projected/e5790b2f-88cb-461c-99ba-d7f0800178e2-kube-api-access-8gwtm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:47.506699 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.506692    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5790b2f-88cb-461c-99ba-d7f0800178e2-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:47.506699 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.506704    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:47.506955 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.506711    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:47.506955 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.506720    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:47.506955 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.506727    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5790b2f-88cb-461c-99ba-d7f0800178e2-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:04:47.697160 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.697063    2577 generic.go:356] "Generic (PLEG): container finished" podID="e5790b2f-88cb-461c-99ba-d7f0800178e2" containerID="cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c" exitCode=0
Jul 29 08:04:47.697160 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.697110    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"
Jul 29 08:04:47.697160 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.697139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv" event={"ID":"e5790b2f-88cb-461c-99ba-d7f0800178e2","Type":"ContainerDied","Data":"cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c"}
Jul 29 08:04:47.697393 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.697176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv" event={"ID":"e5790b2f-88cb-461c-99ba-d7f0800178e2","Type":"ContainerDied","Data":"7da7e265e0dba5a62b00b761804049cb1774edb7da846ba8531c9bbe58937fa7"}
Jul 29 08:04:47.697393 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.697199    2577 scope.go:117] "RemoveContainer" containerID="cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c"
Jul 29 08:04:47.706447 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.706424    2577 scope.go:117] "RemoveContainer" containerID="cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c"
Jul 29 08:04:47.706678 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:04:47.706656    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c\": container with ID starting with cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c not found: ID does not exist" containerID="cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c"
Jul 29 08:04:47.706726 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.706686    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c"} err="failed to get container status \"cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c\": rpc error: code = NotFound desc = could not find container \"cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c\": container with ID starting with cede86eb18a848d57625da0ca1cbcd28aab60efe979d1786bf58ed14711efa3c not found: ID does not exist"
Jul 29 08:04:47.720389 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.720358    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"]
Jul 29 08:04:47.736828 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:47.736796    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/sub-test-model-71e038-kserve-69f44b675-rd7qv"]
Jul 29 08:04:48.789855 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:48.789822    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5790b2f-88cb-461c-99ba-d7f0800178e2" path="/var/lib/kubelet/pods/e5790b2f-88cb-461c-99ba-d7f0800178e2/volumes"
Jul 29 08:04:50.406544 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.406508    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"]
Jul 29 08:04:50.407068 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.407046    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5790b2f-88cb-461c-99ba-d7f0800178e2" containerName="main"
Jul 29 08:04:50.407159 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.407071    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5790b2f-88cb-461c-99ba-d7f0800178e2" containerName="main"
Jul 29 08:04:50.407159 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.407094    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07926ca1-55fa-4de5-8cbd-592a267225e6" containerName="main"
Jul 29 08:04:50.407159 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.407102    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="07926ca1-55fa-4de5-8cbd-592a267225e6" containerName="main"
Jul 29 08:04:50.407312 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.407219    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="07926ca1-55fa-4de5-8cbd-592a267225e6" containerName="main"
Jul 29 08:04:50.407312 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.407235    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5790b2f-88cb-461c-99ba-d7f0800178e2" containerName="main"
Jul 29 08:04:50.411464 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.411439    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.414712 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.414691    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"kube-root-ca.crt\""
Jul 29 08:04:50.414824 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.414781    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"openshift-service-ca.crt\""
Jul 29 08:04:50.415943 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.415927    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"rate-test-model-984886-kserve-self-signed-certs\""
Jul 29 08:04:50.416076 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.416057    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-984886\"/\"default-dockercfg-2fzqx\""
Jul 29 08:04:50.420490 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.420469    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"]
Jul 29 08:04:50.531365 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.531326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-model-cache\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.531553 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.531375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tmp-dir\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.531553 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.531445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-dshm\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.531553 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.531542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btxlr\" (UniqueName: \"kubernetes.io/projected/c1d399c9-f8f8-4728-ae28-1f839397c5b9-kube-api-access-btxlr\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.531660 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.531581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-home\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.531660 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.531600    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tls-certs\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.632864 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.632826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tmp-dir\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.632864 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.632870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-dshm\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633165 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.632922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btxlr\" (UniqueName: \"kubernetes.io/projected/c1d399c9-f8f8-4728-ae28-1f839397c5b9-kube-api-access-btxlr\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633165 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.632952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-home\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633165 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.633033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tls-certs\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633165 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.633097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-model-cache\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633384 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.633321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tmp-dir\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633384 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.633331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-home\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.633458 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.633423    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-model-cache\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.635252 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.635231    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-dshm\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.635423 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.635404    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tls-certs\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.642958 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.642934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btxlr\" (UniqueName: \"kubernetes.io/projected/c1d399c9-f8f8-4728-ae28-1f839397c5b9-kube-api-access-btxlr\") pod \"rate-test-model-984886-kserve-844d56c455-4t846\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") " pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.721978 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.721878    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:04:50.860727 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:50.860685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"]
Jul 29 08:04:50.862895 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:04:50.862867    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1d399c9_f8f8_4728_ae28_1f839397c5b9.slice/crio-52b31ec1362c23a9945c969035ade8abebc10340a4ee1269554bd1a4fe11829e WatchSource:0}: Error finding container 52b31ec1362c23a9945c969035ade8abebc10340a4ee1269554bd1a4fe11829e: Status 404 returned error can't find the container with id 52b31ec1362c23a9945c969035ade8abebc10340a4ee1269554bd1a4fe11829e
Jul 29 08:04:51.717910 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:51.717874    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" event={"ID":"c1d399c9-f8f8-4728-ae28-1f839397c5b9","Type":"ContainerStarted","Data":"4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb"}
Jul 29 08:04:51.718314 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:51.717935    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" event={"ID":"c1d399c9-f8f8-4728-ae28-1f839397c5b9","Type":"ContainerStarted","Data":"52b31ec1362c23a9945c969035ade8abebc10340a4ee1269554bd1a4fe11829e"}
Jul 29 08:04:51.739125 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:51.739078    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" podStartSLOduration=1.739063086 podStartE2EDuration="1.739063086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:04:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:04:51.737115953 +0000 UTC m=+2973.485181798" watchObservedRunningTime="2026-07-29 08:04:51.739063086 +0000 UTC m=+2973.487128927"
Jul 29 08:04:54.390773 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:04:54.390738    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:05:00.722369 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:00.722327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:05:00.722773 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:00.722390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:05:00.734964 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:00.734939    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:05:00.769046 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:00.769014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:05:24.698782 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.698741    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"]
Jul 29 08:05:24.703572 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.703546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.708402 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.708381    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"kube-root-ca.crt\""
Jul 29 08:05:24.708402 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.708394    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"openshift-service-ca.crt\""
Jul 29 08:05:24.708536 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.708414    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"rate-test-model-71e038-kserve-self-signed-certs\""
Jul 29 08:05:24.712919 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.712887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-71e038\"/\"default-dockercfg-w5ttt\""
Jul 29 08:05:24.716205 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.716185    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"]
Jul 29 08:05:24.842057 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.842027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-dshm\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.842229 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.842069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tls-certs\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.842229 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.842105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-home\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.842229 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.842160    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gltq2\" (UniqueName: \"kubernetes.io/projected/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-kube-api-access-gltq2\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.842229 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.842191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tmp-dir\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.842229 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.842212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-model-cache\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943323 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943281    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-home\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943520 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gltq2\" (UniqueName: \"kubernetes.io/projected/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-kube-api-access-gltq2\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943520 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tmp-dir\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943644 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-model-cache\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943644 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943630    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-dshm\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943755 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tls-certs\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943815 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-home\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943815 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-model-cache\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.943979 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.943960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tmp-dir\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.945982 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.945953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-dshm\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.946189 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.946170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tls-certs\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:24.959533 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:24.959462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gltq2\" (UniqueName: \"kubernetes.io/projected/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-kube-api-access-gltq2\") pod \"rate-test-model-71e038-kserve-9569fb4c-h867k\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") " pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:25.012661 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:25.012623    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:25.149015 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:25.148979    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"]
Jul 29 08:05:25.152093 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:05:25.152067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod359b11bc_c861_4b0d_a0b4_7eae1d35bcbf.slice/crio-19f6f4128aba1bbbe222c5229231296a425c0d7da8d4d2f22933b580c3b034c2 WatchSource:0}: Error finding container 19f6f4128aba1bbbe222c5229231296a425c0d7da8d4d2f22933b580c3b034c2: Status 404 returned error can't find the container with id 19f6f4128aba1bbbe222c5229231296a425c0d7da8d4d2f22933b580c3b034c2
Jul 29 08:05:25.868313 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:25.868276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k" event={"ID":"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf","Type":"ContainerStarted","Data":"0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c"}
Jul 29 08:05:25.868714 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:25.868323    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k" event={"ID":"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf","Type":"ContainerStarted","Data":"19f6f4128aba1bbbe222c5229231296a425c0d7da8d4d2f22933b580c3b034c2"}
Jul 29 08:05:25.889952 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:25.889881    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k" podStartSLOduration=1.889864057 podStartE2EDuration="1.889864057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:05:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:05:25.887038585 +0000 UTC m=+3007.635104426" watchObservedRunningTime="2026-07-29 08:05:25.889864057 +0000 UTC m=+3007.637929898"
Jul 29 08:05:28.601530 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:28.601495    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:05:35.013351 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:35.013316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:35.013778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:35.013367    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:35.025816 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:35.025790    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:05:35.925321 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:05:35.925292    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:06:07.134231 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:07.134193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:06:28.794527 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:28.794447    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:06:35.741961 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:35.741928    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"]
Jul 29 08:06:36.592418 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:36.592386    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"]
Jul 29 08:06:48.725456 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:48.725415    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:06:50.760361 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:50.760316    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" podUID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" containerName="main" containerID="cri-o://4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb" gracePeriod=45
Jul 29 08:06:50.764832 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:50.764799    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" podUID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": EOF"
Jul 29 08:06:51.010080 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.010053    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:06:51.116850 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.116767    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-model-cache\") pod \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") "
Jul 29 08:06:51.116850 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.116812    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-home\") pod \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") "
Jul 29 08:06:51.116850 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.116845    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-dshm\") pod \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") "
Jul 29 08:06:51.117116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.116880    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btxlr\" (UniqueName: \"kubernetes.io/projected/c1d399c9-f8f8-4728-ae28-1f839397c5b9-kube-api-access-btxlr\") pod \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") "
Jul 29 08:06:51.117116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.116979    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tmp-dir\") pod \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") "
Jul 29 08:06:51.117116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117012    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tls-certs\") pod \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\" (UID: \"c1d399c9-f8f8-4728-ae28-1f839397c5b9\") "
Jul 29 08:06:51.117116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-home" (OuterVolumeSpecName: "home") pod "c1d399c9-f8f8-4728-ae28-1f839397c5b9" (UID: "c1d399c9-f8f8-4728-ae28-1f839397c5b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.117116 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117096    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "c1d399c9-f8f8-4728-ae28-1f839397c5b9" (UID: "c1d399c9-f8f8-4728-ae28-1f839397c5b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.117391 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117268    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c1d399c9-f8f8-4728-ae28-1f839397c5b9" (UID: "c1d399c9-f8f8-4728-ae28-1f839397c5b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.117391 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117353    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.117391 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117369    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.117391 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.117378    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.118981 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.118954    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-dshm" (OuterVolumeSpecName: "dshm") pod "c1d399c9-f8f8-4728-ae28-1f839397c5b9" (UID: "c1d399c9-f8f8-4728-ae28-1f839397c5b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.119067 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.119042    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d399c9-f8f8-4728-ae28-1f839397c5b9-kube-api-access-btxlr" (OuterVolumeSpecName: "kube-api-access-btxlr") pod "c1d399c9-f8f8-4728-ae28-1f839397c5b9" (UID: "c1d399c9-f8f8-4728-ae28-1f839397c5b9"). InnerVolumeSpecName "kube-api-access-btxlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:06:51.119130 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.119111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c1d399c9-f8f8-4728-ae28-1f839397c5b9" (UID: "c1d399c9-f8f8-4728-ae28-1f839397c5b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:06:51.218259 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.218225    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c1d399c9-f8f8-4728-ae28-1f839397c5b9-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.218259 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.218256    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btxlr\" (UniqueName: \"kubernetes.io/projected/c1d399c9-f8f8-4728-ae28-1f839397c5b9-kube-api-access-btxlr\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.218447 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.218271    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c1d399c9-f8f8-4728-ae28-1f839397c5b9-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.254832 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.254801    2577 generic.go:356] "Generic (PLEG): container finished" podID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" containerID="4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb" exitCode=0
Jul 29 08:06:51.254986 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.254847    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"
Jul 29 08:06:51.254986 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.254864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" event={"ID":"c1d399c9-f8f8-4728-ae28-1f839397c5b9","Type":"ContainerDied","Data":"4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb"}
Jul 29 08:06:51.254986 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.254896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846" event={"ID":"c1d399c9-f8f8-4728-ae28-1f839397c5b9","Type":"ContainerDied","Data":"52b31ec1362c23a9945c969035ade8abebc10340a4ee1269554bd1a4fe11829e"}
Jul 29 08:06:51.254986 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.254941    2577 scope.go:117] "RemoveContainer" containerID="4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb"
Jul 29 08:06:51.264725 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.264706    2577 scope.go:117] "RemoveContainer" containerID="4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb"
Jul 29 08:06:51.265050 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:06:51.265032    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb\": container with ID starting with 4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb not found: ID does not exist" containerID="4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb"
Jul 29 08:06:51.265128 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.265057    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb"} err="failed to get container status \"4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb\": rpc error: code = NotFound desc = could not find container \"4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb\": container with ID starting with 4d08b2609dc46d2b8ba2e8e8ec9b9fa3b948c70877c010ffddf172dff9a3bedb not found: ID does not exist"
Jul 29 08:06:51.284730 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.284704    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"]
Jul 29 08:06:51.296185 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.296163    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-984886/rate-test-model-984886-kserve-844d56c455-4t846"]
Jul 29 08:06:51.610703 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.610657    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k" podUID="359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" containerName="main" containerID="cri-o://0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c" gracePeriod=45
Jul 29 08:06:51.850815 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.850787    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:06:51.923257 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923225    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-model-cache\") pod \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") "
Jul 29 08:06:51.923444 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923276    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-home\") pod \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") "
Jul 29 08:06:51.923444 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923299    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-dshm\") pod \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") "
Jul 29 08:06:51.923444 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923318    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tls-certs\") pod \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") "
Jul 29 08:06:51.923444 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923343    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gltq2\" (UniqueName: \"kubernetes.io/projected/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-kube-api-access-gltq2\") pod \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") "
Jul 29 08:06:51.923444 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923360    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tmp-dir\") pod \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\" (UID: \"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf\") "
Jul 29 08:06:51.923721 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923520    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-model-cache" (OuterVolumeSpecName: "model-cache") pod "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" (UID: "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.923721 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923633    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-home" (OuterVolumeSpecName: "home") pod "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" (UID: "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.923721 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923673    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" (UID: "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.923848 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923830    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.923885 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923853    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.923885 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.923864    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:51.925453 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.925421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" (UID: "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:06:51.925858 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.925842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-dshm" (OuterVolumeSpecName: "dshm") pod "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" (UID: "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:06:51.925943 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:51.925863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-kube-api-access-gltq2" (OuterVolumeSpecName: "kube-api-access-gltq2") pod "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" (UID: "359b11bc-c861-4b0d-a0b4-7eae1d35bcbf"). InnerVolumeSpecName "kube-api-access-gltq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:06:52.025045 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.025003    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gltq2\" (UniqueName: \"kubernetes.io/projected/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-kube-api-access-gltq2\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:52.025045 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.025043    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:52.025045 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.025054    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:52.264191 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.264099    2577 generic.go:356] "Generic (PLEG): container finished" podID="359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" containerID="0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c" exitCode=0
Jul 29 08:06:52.264191 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.264145    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"
Jul 29 08:06:52.264191 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.264179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k" event={"ID":"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf","Type":"ContainerDied","Data":"0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c"}
Jul 29 08:06:52.264488 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.264224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k" event={"ID":"359b11bc-c861-4b0d-a0b4-7eae1d35bcbf","Type":"ContainerDied","Data":"19f6f4128aba1bbbe222c5229231296a425c0d7da8d4d2f22933b580c3b034c2"}
Jul 29 08:06:52.264488 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.264243    2577 scope.go:117] "RemoveContainer" containerID="0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c"
Jul 29 08:06:52.272659 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.272643    2577 scope.go:117] "RemoveContainer" containerID="0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c"
Jul 29 08:06:52.272933 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:06:52.272895    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c\": container with ID starting with 0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c not found: ID does not exist" containerID="0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c"
Jul 29 08:06:52.272997 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.272945    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c"} err="failed to get container status \"0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c\": rpc error: code = NotFound desc = could not find container \"0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c\": container with ID starting with 0c28c817769ed9dfc1ccefaeb9b13ba35545b2c34f1f0a135992322724ba187c not found: ID does not exist"
Jul 29 08:06:52.287498 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.287474    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"]
Jul 29 08:06:52.293251 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.293229    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-71e038/rate-test-model-71e038-kserve-9569fb4c-h867k"]
Jul 29 08:06:52.695390 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695357    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7802-140618635222304"]
Jul 29 08:06:52.695696 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695684    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" containerName="main"
Jul 29 08:06:52.695751 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695697    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" containerName="main"
Jul 29 08:06:52.695751 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695709    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" containerName="main"
Jul 29 08:06:52.695751 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695714    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" containerName="main"
Jul 29 08:06:52.695847 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695764    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" containerName="main"
Jul 29 08:06:52.695847 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.695772    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" containerName="main"
Jul 29 08:06:52.699997 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.699980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:52.704762 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.704712    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-m4vmz\""
Jul 29 08:06:52.709713 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.709691    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7802-140618635222304"]
Jul 29 08:06:52.789748 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.789712    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="359b11bc-c861-4b0d-a0b4-7eae1d35bcbf" path="/var/lib/kubelet/pods/359b11bc-c861-4b0d-a0b4-7eae1d35bcbf/volumes"
Jul 29 08:06:52.790130 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.790116    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d399c9-f8f8-4728-ae28-1f839397c5b9" path="/var/lib/kubelet/pods/c1d399c9-f8f8-4728-ae28-1f839397c5b9/volumes"
Jul 29 08:06:52.833749 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.833720    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp9xd\" (UniqueName: \"kubernetes.io/projected/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a-kube-api-access-pp9xd\") pod \"test-curl-7802-140618635222304\" (UID: \"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a\") " pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:52.934507 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.934470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp9xd\" (UniqueName: \"kubernetes.io/projected/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a-kube-api-access-pp9xd\") pod \"test-curl-7802-140618635222304\" (UID: \"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a\") " pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:52.949400 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:52.949327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp9xd\" (UniqueName: \"kubernetes.io/projected/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a-kube-api-access-pp9xd\") pod \"test-curl-7802-140618635222304\" (UID: \"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a\") " pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:53.011165 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:53.011131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:53.142061 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:53.142002    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7802-140618635222304"]
Jul 29 08:06:53.145761 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:06:53.145732    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6cf6052_b93a_4ebd_a94d_e60ed83b0f2a.slice/crio-874655a0047dc83ac7771652b71fdecc54025a97b4ea595b662235d3d376d368 WatchSource:0}: Error finding container 874655a0047dc83ac7771652b71fdecc54025a97b4ea595b662235d3d376d368: Status 404 returned error can't find the container with id 874655a0047dc83ac7771652b71fdecc54025a97b4ea595b662235d3d376d368
Jul 29 08:06:53.146978 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:53.146961    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 08:06:53.271639 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:53.271559    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7802-140618635222304" event={"ID":"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a","Type":"ContainerStarted","Data":"874655a0047dc83ac7771652b71fdecc54025a97b4ea595b662235d3d376d368"}
Jul 29 08:06:54.277089 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:54.277055    2577 generic.go:356] "Generic (PLEG): container finished" podID="b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" containerID="ac628552f90baede8cfd289dc7678966778739c94786ae4d32f79397726bd366" exitCode=0
Jul 29 08:06:54.277474 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:54.277121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7802-140618635222304" event={"ID":"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a","Type":"ContainerDied","Data":"ac628552f90baede8cfd289dc7678966778739c94786ae4d32f79397726bd366"}
Jul 29 08:06:55.410834 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.410812    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:55.498757 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.498730    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7802-140618635222304_b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a/test-curl-7802-140618635222304/0.log"
Jul 29 08:06:55.554197 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.554103    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp9xd\" (UniqueName: \"kubernetes.io/projected/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a-kube-api-access-pp9xd\") pod \"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a\" (UID: \"b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a\") "
Jul 29 08:06:55.556857 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.556822    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a-kube-api-access-pp9xd" (OuterVolumeSpecName: "kube-api-access-pp9xd") pod "b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" (UID: "b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a"). InnerVolumeSpecName "kube-api-access-pp9xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:06:55.557228 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.557207    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7802-140618635222304"]
Jul 29 08:06:55.561642 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.561621    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7802-140618635222304"]
Jul 29 08:06:55.655339 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:55.655304    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp9xd\" (UniqueName: \"kubernetes.io/projected/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a-kube-api-access-pp9xd\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:06:56.288173 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:56.288144    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="874655a0047dc83ac7771652b71fdecc54025a97b4ea595b662235d3d376d368"
Jul 29 08:06:56.288361 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:56.288180    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:56.299207 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:06:56.299178    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7802-140618635222304\" is forbidden: User \"system:node:ip-10-0-137-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-55.ec2.internal' and this object" podUID="b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" pod="opendatahub/test-curl-7802-140618635222304"
Jul 29 08:06:56.789825 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:06:56.789790    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" path="/var/lib/kubelet/pods/b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a/volumes"
Jul 29 08:07:01.119109 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.119075    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"]
Jul 29 08:07:01.119486 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.119401    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" containerName="test-curl-7802-140618635222304"
Jul 29 08:07:01.119486 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.119413    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" containerName="test-curl-7802-140618635222304"
Jul 29 08:07:01.119486 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.119477    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6cf6052-b93a-4ebd-a94d-e60ed83b0f2a" containerName="test-curl-7802-140618635222304"
Jul 29 08:07:01.122316 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.122299    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.128956 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.128934    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-940406-dockercfg-spt78\""
Jul 29 08:07:01.129128 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.129108    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-940406\""
Jul 29 08:07:01.144961 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.144938    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"]
Jul 29 08:07:01.206824 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.206791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs8pd\" (UniqueName: \"kubernetes.io/projected/dd61c412-17b1-4c85-af7a-0d06829799a7-kube-api-access-hs8pd\") pod \"payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.206988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.206965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd61c412-17b1-4c85-af7a-0d06829799a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.308035 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.307999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd61c412-17b1-4c85-af7a-0d06829799a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.308035 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.308035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8pd\" (UniqueName: \"kubernetes.io/projected/dd61c412-17b1-4c85-af7a-0d06829799a7-kube-api-access-hs8pd\") pod \"payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.308666 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.308645    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd61c412-17b1-4c85-af7a-0d06829799a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.317990 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.317960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs8pd\" (UniqueName: \"kubernetes.io/projected/dd61c412-17b1-4c85-af7a-0d06829799a7-kube-api-access-hs8pd\") pod \"payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.432042 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.432008    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:01.563974 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:01.563942    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"]
Jul 29 08:07:01.566135 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:07:01.566109    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd61c412_17b1_4c85_af7a_0d06829799a7.slice/crio-8c26683ed411fba0aa5477cae3810a8e97cb2860812d722c8e4956cb029ba51f WatchSource:0}: Error finding container 8c26683ed411fba0aa5477cae3810a8e97cb2860812d722c8e4956cb029ba51f: Status 404 returned error can't find the container with id 8c26683ed411fba0aa5477cae3810a8e97cb2860812d722c8e4956cb029ba51f
Jul 29 08:07:02.319020 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:02.318970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp" event={"ID":"dd61c412-17b1-4c85-af7a-0d06829799a7","Type":"ContainerStarted","Data":"8c26683ed411fba0aa5477cae3810a8e97cb2860812d722c8e4956cb029ba51f"}
Jul 29 08:07:03.324988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:03.324951    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp" event={"ID":"dd61c412-17b1-4c85-af7a-0d06829799a7","Type":"ContainerStarted","Data":"98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4"}
Jul 29 08:07:03.325440 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:03.325078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:03.346882 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:03.346830    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp" podStartSLOduration=0.725528328 podStartE2EDuration="2.346817134s" totalImagesPullingTime="1.621288806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:07:03.344983719 +0000 UTC m=+3105.093049562" watchObservedRunningTime="2026-07-29 08:07:03.346817134 +0000 UTC m=+3105.094882955"
Jul 29 08:07:07.689926 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:07.689878    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:07:14.332122 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:14.332091    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:07:20.809967 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.809930    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"]
Jul 29 08:07:20.814884 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.814857    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.817688 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.817657    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-f1dc98-serving-cert\""
Jul 29 08:07:20.817823 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.817683    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-t9lbj\""
Jul 29 08:07:20.817823 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.817683    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 08:07:20.819317 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.819264    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 08:07:20.821651 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.821632    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"]
Jul 29 08:07:20.881830 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.881791    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4rwz\" (UniqueName: \"kubernetes.io/projected/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-kube-api-access-c4rwz\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.882038 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.881926    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.882038 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.881974    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.983343 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.983294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.983343 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.983349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.983617 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.983391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4rwz\" (UniqueName: \"kubernetes.io/projected/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-kube-api-access-c4rwz\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.983617 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:07:20.983510    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-serving-cert: secret "maas-api-e2e-ipp-b-f1dc98-serving-cert" not found
Jul 29 08:07:20.983617 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:07:20.983583    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls podName:e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b nodeName:}" failed. No retries permitted until 2026-07-29 08:07:21.483567326 +0000 UTC m=+3123.231633149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls") pod "maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" (UID: "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b") : secret "maas-api-e2e-ipp-b-f1dc98-serving-cert" not found
Jul 29 08:07:20.984084 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.984060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:20.997938 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:20.997892    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4rwz\" (UniqueName: \"kubernetes.io/projected/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-kube-api-access-c4rwz\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:21.488689 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:21.488652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:21.488864 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:07:21.488794    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-serving-cert: secret "maas-api-e2e-ipp-b-f1dc98-serving-cert" not found
Jul 29 08:07:21.488948 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:07:21.488871    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls podName:e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b nodeName:}" failed. No retries permitted until 2026-07-29 08:07:22.488855594 +0000 UTC m=+3124.236921413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls") pod "maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" (UID: "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b") : secret "maas-api-e2e-ipp-b-f1dc98-serving-cert" not found
Jul 29 08:07:22.498450 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:22.498413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:22.500936 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:22.500890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:22.626321 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:22.626274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:22.762790 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:22.762764    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"]
Jul 29 08:07:22.764860 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:07:22.764834    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7ef5bcf_693c_4052_bb6e_6bfca5b53e9b.slice/crio-3151e1fb37622a6d09177539240a95e500c2bd02b20260f4351890092360e9a2 WatchSource:0}: Error finding container 3151e1fb37622a6d09177539240a95e500c2bd02b20260f4351890092360e9a2: Status 404 returned error can't find the container with id 3151e1fb37622a6d09177539240a95e500c2bd02b20260f4351890092360e9a2
Jul 29 08:07:23.415388 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:23.415358    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" event={"ID":"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b","Type":"ContainerStarted","Data":"3151e1fb37622a6d09177539240a95e500c2bd02b20260f4351890092360e9a2"}
Jul 29 08:07:26.430306 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:26.430269    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" event={"ID":"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b","Type":"ContainerStarted","Data":"79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0"}
Jul 29 08:07:26.430676 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:26.430349    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:26.450410 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:26.450359    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" podStartSLOduration=3.005417979 podStartE2EDuration="6.450342876s" totalImagesPullingTime="3.444924897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:07:26.449750504 +0000 UTC m=+3128.197816342" watchObservedRunningTime="2026-07-29 08:07:26.450342876 +0000 UTC m=+3128.198408721"
Jul 29 08:07:29.484412 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:29.484375    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:07:32.440867 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:32.440839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:07:50.046040 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.046001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"]
Jul 29 08:07:50.049852 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.049831    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.053954 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.053930    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-940406\"/\"ipp-route-940406-kserve-self-signed-certs\""
Jul 29 08:07:50.053954 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.053954    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-940406\"/\"kube-root-ca.crt\""
Jul 29 08:07:50.054146 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.053973    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-940406\"/\"default-dockercfg-kpgsg\""
Jul 29 08:07:50.054146 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.053931    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-940406\"/\"openshift-service-ca.crt\""
Jul 29 08:07:50.056851 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.056829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"]
Jul 29 08:07:50.150837 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.150803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hndh8\" (UniqueName: \"kubernetes.io/projected/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-kube-api-access-hndh8\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.151036 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.150871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-dshm\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.151036 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.150896    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-model-cache\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.151036 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.150989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tmp-dir\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.151036 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.151011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-home\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.151244 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.151100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tls-certs\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.251798 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.251760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hndh8\" (UniqueName: \"kubernetes.io/projected/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-kube-api-access-hndh8\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.251993 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.251817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-dshm\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.251993 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.251950    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-model-cache\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.252084 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.252021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tmp-dir\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.252084 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.252050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-home\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.252170 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.252106    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tls-certs\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.252346 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.252323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-model-cache\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.252401 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.252333    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tmp-dir\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.252458 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.252418    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-home\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.254101 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.254077    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-dshm\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.254412 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.254395    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tls-certs\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.262549 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.262530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hndh8\" (UniqueName: \"kubernetes.io/projected/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-kube-api-access-hndh8\") pod \"ipp-route-940406-kserve-dbd9b977f-pln8s\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") " pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.362559 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.362468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:07:50.499733 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.499707    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"]
Jul 29 08:07:50.501183 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:07:50.501162    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc3c297e_d9f8_4769_b9e8_b3335472ffc3.slice/crio-75d41bea801ad6f5483780d8e0bdc00a16ecfbd9341bc19e4ef150e9ee62c495 WatchSource:0}: Error finding container 75d41bea801ad6f5483780d8e0bdc00a16ecfbd9341bc19e4ef150e9ee62c495: Status 404 returned error can't find the container with id 75d41bea801ad6f5483780d8e0bdc00a16ecfbd9341bc19e4ef150e9ee62c495
Jul 29 08:07:50.537556 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:50.537524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s" event={"ID":"fc3c297e-d9f8-4769-b9e8-b3335472ffc3","Type":"ContainerStarted","Data":"75d41bea801ad6f5483780d8e0bdc00a16ecfbd9341bc19e4ef150e9ee62c495"}
Jul 29 08:07:51.543211 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:51.543164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s" event={"ID":"fc3c297e-d9f8-4769-b9e8-b3335472ffc3","Type":"ContainerStarted","Data":"9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508"}
Jul 29 08:07:51.564877 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:51.564827    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s" podStartSLOduration=1.5648120749999999 podStartE2EDuration="1.564812075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:07:51.562790476 +0000 UTC m=+3153.310856318" watchObservedRunningTime="2026-07-29 08:07:51.564812075 +0000 UTC m=+3153.312877916"
Jul 29 08:07:54.113455 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:07:54.113417    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:08:00.363357 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:00.363310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:08:00.363758 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:00.363379    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:08:00.376119 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:00.376092    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:08:00.592429 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:00.592391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:08:26.489334 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:26.489292    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:08:43.742233 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.742198    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"]
Jul 29 08:08:43.746062 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.746028    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.749111 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.749090    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f73166-serving-cert\""
Jul 29 08:08:43.775491 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.775358    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"]
Jul 29 08:08:43.820037 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.819998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkzm9\" (UniqueName: \"kubernetes.io/projected/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-kube-api-access-jkzm9\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.820219 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.820053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.820219 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.820111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.921016 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.920962    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkzm9\" (UniqueName: \"kubernetes.io/projected/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-kube-api-access-jkzm9\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.921195 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.921036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.921195 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.921065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.921195 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:08:43.921159    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-serving-cert: secret "maas-api-e2e-ipp-cleanup-f73166-serving-cert" not found
Jul 29 08:08:43.921363 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:08:43.921228    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls podName:6b94cb99-c548-40d9-85b5-2fa4e040c9c7 nodeName:}" failed. No retries permitted until 2026-07-29 08:08:44.421210131 +0000 UTC m=+3206.169275953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" (UID: "6b94cb99-c548-40d9-85b5-2fa4e040c9c7") : secret "maas-api-e2e-ipp-cleanup-f73166-serving-cert" not found
Jul 29 08:08:43.921740 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.921716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:43.932471 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:43.932448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkzm9\" (UniqueName: \"kubernetes.io/projected/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-kube-api-access-jkzm9\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:44.360038 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.360001    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"]
Jul 29 08:08:44.363791 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.363770    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.366991 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.366974    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-f73166-dockercfg-5cqpz\""
Jul 29 08:08:44.367191 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.367168    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-f73166\""
Jul 29 08:08:44.374975 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.374956    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"]
Jul 29 08:08:44.389007 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.388985    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"]
Jul 29 08:08:44.392294 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.392275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.402825 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.402801    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"]
Jul 29 08:08:44.425486 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.425460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mdh5\" (UniqueName: \"kubernetes.io/projected/28c3b6d9-7b54-4659-a118-11ec099329e3-kube-api-access-6mdh5\") pod \"payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.425585 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.425521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:44.425634 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.425584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfqhh\" (UniqueName: \"kubernetes.io/projected/65a61b32-70ec-471c-b82f-9d82844d3c37-kube-api-access-kfqhh\") pod \"payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.425692 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.425629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a61b32-70ec-471c-b82f-9d82844d3c37-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.425692 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.425663    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c3b6d9-7b54-4659-a118-11ec099329e3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.427802 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.427784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:44.526112 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.526077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfqhh\" (UniqueName: \"kubernetes.io/projected/65a61b32-70ec-471c-b82f-9d82844d3c37-kube-api-access-kfqhh\") pod \"payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.526289 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.526119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a61b32-70ec-471c-b82f-9d82844d3c37-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.526289 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.526156    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c3b6d9-7b54-4659-a118-11ec099329e3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.526289 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.526186    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mdh5\" (UniqueName: \"kubernetes.io/projected/28c3b6d9-7b54-4659-a118-11ec099329e3-kube-api-access-6mdh5\") pod \"payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.526991 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.526968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c3b6d9-7b54-4659-a118-11ec099329e3-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.527094 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.526967    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a61b32-70ec-471c-b82f-9d82844d3c37-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.535496 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.535473    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mdh5\" (UniqueName: \"kubernetes.io/projected/28c3b6d9-7b54-4659-a118-11ec099329e3-kube-api-access-6mdh5\") pod \"payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.537278 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.537258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfqhh\" (UniqueName: \"kubernetes.io/projected/65a61b32-70ec-471c-b82f-9d82844d3c37-kube-api-access-kfqhh\") pod \"payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.656478 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.656444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:44.674756 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.674731    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:44.701777 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.701741    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:44.847828 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.847802    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"]
Jul 29 08:08:44.849358 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:08:44.849328    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a61b32_70ec_471c_b82f_9d82844d3c37.slice/crio-2bc452197447415b9d2fdf12f2035136604bb2c223be941e139c05a0b1127b0e WatchSource:0}: Error finding container 2bc452197447415b9d2fdf12f2035136604bb2c223be941e139c05a0b1127b0e: Status 404 returned error can't find the container with id 2bc452197447415b9d2fdf12f2035136604bb2c223be941e139c05a0b1127b0e
Jul 29 08:08:44.866651 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:08:44.866629    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c3b6d9_7b54_4659_a118_11ec099329e3.slice/crio-89c8aa8b854cb8555b794d4033bbc14fcf6637ddc65051d7d7c42a638b673ee2 WatchSource:0}: Error finding container 89c8aa8b854cb8555b794d4033bbc14fcf6637ddc65051d7d7c42a638b673ee2: Status 404 returned error can't find the container with id 89c8aa8b854cb8555b794d4033bbc14fcf6637ddc65051d7d7c42a638b673ee2
Jul 29 08:08:44.871362 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:44.871339    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"]
Jul 29 08:08:45.021344 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.021304    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"]
Jul 29 08:08:45.023822 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:08:45.023793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b94cb99_c548_40d9_85b5_2fa4e040c9c7.slice/crio-185c0e980d9ffb30620c414efed0d238ccaf08bdedbb99ef9e86ecb8beff4fe0 WatchSource:0}: Error finding container 185c0e980d9ffb30620c414efed0d238ccaf08bdedbb99ef9e86ecb8beff4fe0: Status 404 returned error can't find the container with id 185c0e980d9ffb30620c414efed0d238ccaf08bdedbb99ef9e86ecb8beff4fe0
Jul 29 08:08:45.789840 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.789738    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl" event={"ID":"65a61b32-70ec-471c-b82f-9d82844d3c37","Type":"ContainerStarted","Data":"65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d"}
Jul 29 08:08:45.789840 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.789792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl" event={"ID":"65a61b32-70ec-471c-b82f-9d82844d3c37","Type":"ContainerStarted","Data":"2bc452197447415b9d2fdf12f2035136604bb2c223be941e139c05a0b1127b0e"}
Jul 29 08:08:45.790119 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.789929    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:08:45.791522 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.791492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp" event={"ID":"28c3b6d9-7b54-4659-a118-11ec099329e3","Type":"ContainerStarted","Data":"62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7"}
Jul 29 08:08:45.791678 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.791537    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:45.791678 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.791553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp" event={"ID":"28c3b6d9-7b54-4659-a118-11ec099329e3","Type":"ContainerStarted","Data":"89c8aa8b854cb8555b794d4033bbc14fcf6637ddc65051d7d7c42a638b673ee2"}
Jul 29 08:08:45.793000 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.792975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" event={"ID":"6b94cb99-c548-40d9-85b5-2fa4e040c9c7","Type":"ContainerStarted","Data":"83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9"}
Jul 29 08:08:45.793117 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.793016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" event={"ID":"6b94cb99-c548-40d9-85b5-2fa4e040c9c7","Type":"ContainerStarted","Data":"185c0e980d9ffb30620c414efed0d238ccaf08bdedbb99ef9e86ecb8beff4fe0"}
Jul 29 08:08:45.793117 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.793105    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:45.810344 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.810290    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl" podStartSLOduration=1.810274659 podStartE2EDuration="1.810274659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:08:45.808091157 +0000 UTC m=+3207.556157001" watchObservedRunningTime="2026-07-29 08:08:45.810274659 +0000 UTC m=+3207.558340501"
Jul 29 08:08:45.835970 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.835917    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp" podStartSLOduration=1.83588518 podStartE2EDuration="1.83588518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:08:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:08:45.835185304 +0000 UTC m=+3207.583251142" watchObservedRunningTime="2026-07-29 08:08:45.83588518 +0000 UTC m=+3207.583951087"
Jul 29 08:08:45.857462 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:45.857413    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" podStartSLOduration=2.342613817 podStartE2EDuration="2.857395916s" totalImagesPullingTime="514.782099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:08:45.856124619 +0000 UTC m=+3207.604190484" watchObservedRunningTime="2026-07-29 08:08:45.857395916 +0000 UTC m=+3207.605461758"
Jul 29 08:08:51.802398 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:51.802363    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:08:51.856176 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:51.856139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:08:56.799792 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:56.799762    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:08:56.800225 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:08:56.799880    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:09:07.390354 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.390319    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"]
Jul 29 08:09:07.391153 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.390527    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" podUID="6b94cb99-c548-40d9-85b5-2fa4e040c9c7" containerName="maas-api" containerID="cri-o://83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9" gracePeriod=30
Jul 29 08:09:07.447917 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.447872    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"]
Jul 29 08:09:07.448160 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.448134    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp" podUID="28c3b6d9-7b54-4659-a118-11ec099329e3" containerName="payload-processing" containerID="cri-o://62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7" gracePeriod=30
Jul 29 08:09:07.455219 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.455172    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"]
Jul 29 08:09:07.455470 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.455422    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl" podUID="65a61b32-70ec-471c-b82f-9d82844d3c37" containerName="payload-pre-processing" containerID="cri-o://65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d" gracePeriod=30
Jul 29 08:09:07.712001 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.711968    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:09:07.714923 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.714888    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:09:07.737282 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.737255    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:09:07.832583 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832546    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mdh5\" (UniqueName: \"kubernetes.io/projected/28c3b6d9-7b54-4659-a118-11ec099329e3-kube-api-access-6mdh5\") pod \"28c3b6d9-7b54-4659-a118-11ec099329e3\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") "
Jul 29 08:09:07.832778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls\") pod \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") "
Jul 29 08:09:07.832778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832629    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c3b6d9-7b54-4659-a118-11ec099329e3-plugins-config-volume\") pod \"28c3b6d9-7b54-4659-a118-11ec099329e3\" (UID: \"28c3b6d9-7b54-4659-a118-11ec099329e3\") "
Jul 29 08:09:07.832778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832675    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a61b32-70ec-471c-b82f-9d82844d3c37-plugins-config-volume\") pod \"65a61b32-70ec-471c-b82f-9d82844d3c37\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") "
Jul 29 08:09:07.832778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832739    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfqhh\" (UniqueName: \"kubernetes.io/projected/65a61b32-70ec-471c-b82f-9d82844d3c37-kube-api-access-kfqhh\") pod \"65a61b32-70ec-471c-b82f-9d82844d3c37\" (UID: \"65a61b32-70ec-471c-b82f-9d82844d3c37\") "
Jul 29 08:09:07.832778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832759    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkzm9\" (UniqueName: \"kubernetes.io/projected/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-kube-api-access-jkzm9\") pod \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") "
Jul 29 08:09:07.832778 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.832777    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-openshift-service-ca\") pod \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\" (UID: \"6b94cb99-c548-40d9-85b5-2fa4e040c9c7\") "
Jul 29 08:09:07.833153 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.833098    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28c3b6d9-7b54-4659-a118-11ec099329e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "28c3b6d9-7b54-4659-a118-11ec099329e3" (UID: "28c3b6d9-7b54-4659-a118-11ec099329e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:09:07.833153 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.833122    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65a61b32-70ec-471c-b82f-9d82844d3c37-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "65a61b32-70ec-471c-b82f-9d82844d3c37" (UID: "65a61b32-70ec-471c-b82f-9d82844d3c37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:09:07.833406 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.833382    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b94cb99-c548-40d9-85b5-2fa4e040c9c7" (UID: "6b94cb99-c548-40d9-85b5-2fa4e040c9c7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:09:07.834977 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.834951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65a61b32-70ec-471c-b82f-9d82844d3c37-kube-api-access-kfqhh" (OuterVolumeSpecName: "kube-api-access-kfqhh") pod "65a61b32-70ec-471c-b82f-9d82844d3c37" (UID: "65a61b32-70ec-471c-b82f-9d82844d3c37"). InnerVolumeSpecName "kube-api-access-kfqhh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:09:07.834977 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.834956    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c3b6d9-7b54-4659-a118-11ec099329e3-kube-api-access-6mdh5" (OuterVolumeSpecName: "kube-api-access-6mdh5") pod "28c3b6d9-7b54-4659-a118-11ec099329e3" (UID: "28c3b6d9-7b54-4659-a118-11ec099329e3"). InnerVolumeSpecName "kube-api-access-6mdh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:09:07.835299 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.835276    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b94cb99-c548-40d9-85b5-2fa4e040c9c7" (UID: "6b94cb99-c548-40d9-85b5-2fa4e040c9c7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:09:07.835299 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.835282    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-kube-api-access-jkzm9" (OuterVolumeSpecName: "kube-api-access-jkzm9") pod "6b94cb99-c548-40d9-85b5-2fa4e040c9c7" (UID: "6b94cb99-c548-40d9-85b5-2fa4e040c9c7"). InnerVolumeSpecName "kube-api-access-jkzm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:09:07.902610 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.902518    2577 generic.go:356] "Generic (PLEG): container finished" podID="65a61b32-70ec-471c-b82f-9d82844d3c37" containerID="65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d" exitCode=0
Jul 29 08:09:07.902610 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.902563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"
Jul 29 08:09:07.902610 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.902599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl" event={"ID":"65a61b32-70ec-471c-b82f-9d82844d3c37","Type":"ContainerDied","Data":"65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d"}
Jul 29 08:09:07.902895 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.902649    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl" event={"ID":"65a61b32-70ec-471c-b82f-9d82844d3c37","Type":"ContainerDied","Data":"2bc452197447415b9d2fdf12f2035136604bb2c223be941e139c05a0b1127b0e"}
Jul 29 08:09:07.902895 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.902671    2577 scope.go:117] "RemoveContainer" containerID="65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d"
Jul 29 08:09:07.903814 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.903788    2577 generic.go:356] "Generic (PLEG): container finished" podID="28c3b6d9-7b54-4659-a118-11ec099329e3" containerID="62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7" exitCode=0
Jul 29 08:09:07.903939 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.903836    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"
Jul 29 08:09:07.903939 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.903881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp" event={"ID":"28c3b6d9-7b54-4659-a118-11ec099329e3","Type":"ContainerDied","Data":"62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7"}
Jul 29 08:09:07.904047 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.903963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp" event={"ID":"28c3b6d9-7b54-4659-a118-11ec099329e3","Type":"ContainerDied","Data":"89c8aa8b854cb8555b794d4033bbc14fcf6637ddc65051d7d7c42a638b673ee2"}
Jul 29 08:09:07.905402 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.905373    2577 generic.go:356] "Generic (PLEG): container finished" podID="6b94cb99-c548-40d9-85b5-2fa4e040c9c7" containerID="83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9" exitCode=0
Jul 29 08:09:07.905510 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.905420    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"
Jul 29 08:09:07.905510 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.905418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" event={"ID":"6b94cb99-c548-40d9-85b5-2fa4e040c9c7","Type":"ContainerDied","Data":"83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9"}
Jul 29 08:09:07.905620 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.905542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6" event={"ID":"6b94cb99-c548-40d9-85b5-2fa4e040c9c7","Type":"ContainerDied","Data":"185c0e980d9ffb30620c414efed0d238ccaf08bdedbb99ef9e86ecb8beff4fe0"}
Jul 29 08:09:07.913751 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.913734    2577 scope.go:117] "RemoveContainer" containerID="65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d"
Jul 29 08:09:07.914087 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:09:07.914063    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d\": container with ID starting with 65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d not found: ID does not exist" containerID="65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d"
Jul 29 08:09:07.914205 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.914095    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d"} err="failed to get container status \"65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d\": rpc error: code = NotFound desc = could not find container \"65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d\": container with ID starting with 65d9ef35db09a8048f624c2ba9223e60c53c7fe47baaf2563db3263f0dbc0a8d not found: ID does not exist"
Jul 29 08:09:07.914205 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.914121    2577 scope.go:117] "RemoveContainer" containerID="62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7"
Jul 29 08:09:07.923296 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.923277    2577 scope.go:117] "RemoveContainer" containerID="62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7"
Jul 29 08:09:07.923603 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:09:07.923584    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7\": container with ID starting with 62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7 not found: ID does not exist" containerID="62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7"
Jul 29 08:09:07.923656 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.923608    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7"} err="failed to get container status \"62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7\": rpc error: code = NotFound desc = could not find container \"62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7\": container with ID starting with 62edfe2f5401b0969ca482a12ecb0d2c643de45fb2619bf19765236f4f2a19b7 not found: ID does not exist"
Jul 29 08:09:07.923656 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.923623    2577 scope.go:117] "RemoveContainer" containerID="83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9"
Jul 29 08:09:07.933148 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933121    2577 scope.go:117] "RemoveContainer" containerID="83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9"
Jul 29 08:09:07.933426 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:09:07.933404    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9\": container with ID starting with 83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9 not found: ID does not exist" containerID="83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9"
Jul 29 08:09:07.933505 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933434    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9"} err="failed to get container status \"83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9\": rpc error: code = NotFound desc = could not find container \"83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9\": container with ID starting with 83dfb71e521b169899bcb30a5733b1dc8afda4fef3c96a496def0660d2e318a9 not found: ID does not exist"
Jul 29 08:09:07.933505 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933477    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfqhh\" (UniqueName: \"kubernetes.io/projected/65a61b32-70ec-471c-b82f-9d82844d3c37-kube-api-access-kfqhh\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.933505 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933496    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkzm9\" (UniqueName: \"kubernetes.io/projected/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-kube-api-access-jkzm9\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.933621 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933507    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-openshift-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.933621 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933518    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mdh5\" (UniqueName: \"kubernetes.io/projected/28c3b6d9-7b54-4659-a118-11ec099329e3-kube-api-access-6mdh5\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.933621 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933534    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b94cb99-c548-40d9-85b5-2fa4e040c9c7-maas-api-tls\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.933621 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933543    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/28c3b6d9-7b54-4659-a118-11ec099329e3-plugins-config-volume\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.933621 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.933551    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/65a61b32-70ec-471c-b82f-9d82844d3c37-plugins-config-volume\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:07.934599 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.934581    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"]
Jul 29 08:09:07.939708 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.939688    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-f73166-64f7b4dcbd-x9sdl"]
Jul 29 08:09:07.953436 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.953409    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"]
Jul 29 08:09:07.957943 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.957922    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-f73166-7cb8d95db6-mv4xp"]
Jul 29 08:09:07.970400 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.970378    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"]
Jul 29 08:09:07.972324 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:07.972302    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f73166-5dbb5cfcc-plhc6"]
Jul 29 08:09:08.790872 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:08.790795    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c3b6d9-7b54-4659-a118-11ec099329e3" path="/var/lib/kubelet/pods/28c3b6d9-7b54-4659-a118-11ec099329e3/volumes"
Jul 29 08:09:08.791250 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:08.791164    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65a61b32-70ec-471c-b82f-9d82844d3c37" path="/var/lib/kubelet/pods/65a61b32-70ec-471c-b82f-9d82844d3c37/volumes"
Jul 29 08:09:08.791528 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:08.791513    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b94cb99-c548-40d9-85b5-2fa4e040c9c7" path="/var/lib/kubelet/pods/6b94cb99-c548-40d9-85b5-2fa4e040c9c7/volumes"
Jul 29 08:09:15.297865 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:15.297773    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:09:18.628704 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.628669    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"]
Jul 29 08:09:18.629100 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629030    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b94cb99-c548-40d9-85b5-2fa4e040c9c7" containerName="maas-api"
Jul 29 08:09:18.629100 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629043    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b94cb99-c548-40d9-85b5-2fa4e040c9c7" containerName="maas-api"
Jul 29 08:09:18.629100 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629055    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c3b6d9-7b54-4659-a118-11ec099329e3" containerName="payload-processing"
Jul 29 08:09:18.629100 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629060    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c3b6d9-7b54-4659-a118-11ec099329e3" containerName="payload-processing"
Jul 29 08:09:18.629100 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629080    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65a61b32-70ec-471c-b82f-9d82844d3c37" containerName="payload-pre-processing"
Jul 29 08:09:18.629100 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629086    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="65a61b32-70ec-471c-b82f-9d82844d3c37" containerName="payload-pre-processing"
Jul 29 08:09:18.629300 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629133    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="65a61b32-70ec-471c-b82f-9d82844d3c37" containerName="payload-pre-processing"
Jul 29 08:09:18.629300 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629140    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c3b6d9-7b54-4659-a118-11ec099329e3" containerName="payload-processing"
Jul 29 08:09:18.629300 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.629146    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b94cb99-c548-40d9-85b5-2fa4e040c9c7" containerName="maas-api"
Jul 29 08:09:18.634130 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.634110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:18.637201 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.637180    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2z4n9\""
Jul 29 08:09:18.642225 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.642197    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"]
Jul 29 08:09:18.737349 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.737300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26beaa83-7bb5-4d89-a9c3-a02e336f5e42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2\" (UID: \"26beaa83-7bb5-4d89-a9c3-a02e336f5e42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:18.838490 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.838446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26beaa83-7bb5-4d89-a9c3-a02e336f5e42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2\" (UID: \"26beaa83-7bb5-4d89-a9c3-a02e336f5e42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:18.839104 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.839082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26beaa83-7bb5-4d89-a9c3-a02e336f5e42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2\" (UID: \"26beaa83-7bb5-4d89-a9c3-a02e336f5e42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:18.945989 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:18.945962    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:19.307841 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:19.307763    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"]
Jul 29 08:09:19.310807 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:09:19.310779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26beaa83_7bb5_4d89_a9c3_a02e336f5e42.slice/crio-4cb0b1693eb2eb10b7bdce8c890dfd78728165b711d632a1f6b3589b5ec49cd3 WatchSource:0}: Error finding container 4cb0b1693eb2eb10b7bdce8c890dfd78728165b711d632a1f6b3589b5ec49cd3: Status 404 returned error can't find the container with id 4cb0b1693eb2eb10b7bdce8c890dfd78728165b711d632a1f6b3589b5ec49cd3
Jul 29 08:09:19.970925 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:19.970868    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2" event={"ID":"26beaa83-7bb5-4d89-a9c3-a02e336f5e42","Type":"ContainerStarted","Data":"4cb0b1693eb2eb10b7bdce8c890dfd78728165b711d632a1f6b3589b5ec49cd3"}
Jul 29 08:09:21.982499 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:21.982414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2" event={"ID":"26beaa83-7bb5-4d89-a9c3-a02e336f5e42","Type":"ContainerStarted","Data":"6ef55bd40e5ada1612bbc9d1387b5ec08e0129e2a9b2fba9943def4cf5487d81"}
Jul 29 08:09:22.010209 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:22.010156    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2" podStartSLOduration=1.911361015 podStartE2EDuration="4.010141623s" totalImagesPullingTime="2.098780608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:09:22.009091845 +0000 UTC m=+3243.757157707" watchObservedRunningTime="2026-07-29 08:09:22.010141623 +0000 UTC m=+3243.758207465"
Jul 29 08:09:22.988267 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:22.988227    2577 generic.go:356] "Generic (PLEG): container finished" podID="26beaa83-7bb5-4d89-a9c3-a02e336f5e42" containerID="6ef55bd40e5ada1612bbc9d1387b5ec08e0129e2a9b2fba9943def4cf5487d81" exitCode=0
Jul 29 08:09:22.988669 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:22.988311    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2" event={"ID":"26beaa83-7bb5-4d89-a9c3-a02e336f5e42","Type":"ContainerDied","Data":"6ef55bd40e5ada1612bbc9d1387b5ec08e0129e2a9b2fba9943def4cf5487d81"}
Jul 29 08:09:24.125975 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.125949    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:24.288139 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.288046    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26beaa83-7bb5-4d89-a9c3-a02e336f5e42-maas-api-service-ca\") pod \"26beaa83-7bb5-4d89-a9c3-a02e336f5e42\" (UID: \"26beaa83-7bb5-4d89-a9c3-a02e336f5e42\") "
Jul 29 08:09:24.288463 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.288434    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26beaa83-7bb5-4d89-a9c3-a02e336f5e42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26beaa83-7bb5-4d89-a9c3-a02e336f5e42" (UID: "26beaa83-7bb5-4d89-a9c3-a02e336f5e42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:09:24.389211 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.389179    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26beaa83-7bb5-4d89-a9c3-a02e336f5e42-maas-api-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:24.998336 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.998307    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"
Jul 29 08:09:24.998531 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.998345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2" event={"ID":"26beaa83-7bb5-4d89-a9c3-a02e336f5e42","Type":"ContainerDied","Data":"4cb0b1693eb2eb10b7bdce8c890dfd78728165b711d632a1f6b3589b5ec49cd3"}
Jul 29 08:09:24.998531 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:24.998405    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cb0b1693eb2eb10b7bdce8c890dfd78728165b711d632a1f6b3589b5ec49cd3"
Jul 29 08:09:34.288446 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:34.288407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:09:36.163602 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.163561    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"]
Jul 29 08:09:36.164059 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.163800    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp" podUID="dd61c412-17b1-4c85-af7a-0d06829799a7" containerName="payload-pre-processing" containerID="cri-o://98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4" gracePeriod=30
Jul 29 08:09:36.367137 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.367106    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"]
Jul 29 08:09:36.374048 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.372013    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-940406-jbkj2"]
Jul 29 08:09:36.418780 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.418723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:09:36.503484 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.503448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs8pd\" (UniqueName: \"kubernetes.io/projected/dd61c412-17b1-4c85-af7a-0d06829799a7-kube-api-access-hs8pd\") pod \"dd61c412-17b1-4c85-af7a-0d06829799a7\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") "
Jul 29 08:09:36.503657 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.503579    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd61c412-17b1-4c85-af7a-0d06829799a7-plugins-config-volume\") pod \"dd61c412-17b1-4c85-af7a-0d06829799a7\" (UID: \"dd61c412-17b1-4c85-af7a-0d06829799a7\") "
Jul 29 08:09:36.504003 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.503974    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd61c412-17b1-4c85-af7a-0d06829799a7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd61c412-17b1-4c85-af7a-0d06829799a7" (UID: "dd61c412-17b1-4c85-af7a-0d06829799a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:09:36.505529 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.505503    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd61c412-17b1-4c85-af7a-0d06829799a7-kube-api-access-hs8pd" (OuterVolumeSpecName: "kube-api-access-hs8pd") pod "dd61c412-17b1-4c85-af7a-0d06829799a7" (UID: "dd61c412-17b1-4c85-af7a-0d06829799a7"). InnerVolumeSpecName "kube-api-access-hs8pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:09:36.605169 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.605138    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs8pd\" (UniqueName: \"kubernetes.io/projected/dd61c412-17b1-4c85-af7a-0d06829799a7-kube-api-access-hs8pd\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:36.605169 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.605166    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd61c412-17b1-4c85-af7a-0d06829799a7-plugins-config-volume\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:36.790675 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:36.790594    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26beaa83-7bb5-4d89-a9c3-a02e336f5e42" path="/var/lib/kubelet/pods/26beaa83-7bb5-4d89-a9c3-a02e336f5e42/volumes"
Jul 29 08:09:37.054405 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.054309    2577 generic.go:356] "Generic (PLEG): container finished" podID="dd61c412-17b1-4c85-af7a-0d06829799a7" containerID="98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4" exitCode=0
Jul 29 08:09:37.054405 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.054353    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"
Jul 29 08:09:37.054405 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.054361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp" event={"ID":"dd61c412-17b1-4c85-af7a-0d06829799a7","Type":"ContainerDied","Data":"98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4"}
Jul 29 08:09:37.054663 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.054411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp" event={"ID":"dd61c412-17b1-4c85-af7a-0d06829799a7","Type":"ContainerDied","Data":"8c26683ed411fba0aa5477cae3810a8e97cb2860812d722c8e4956cb029ba51f"}
Jul 29 08:09:37.054663 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.054432    2577 scope.go:117] "RemoveContainer" containerID="98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4"
Jul 29 08:09:37.063117 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.063090    2577 scope.go:117] "RemoveContainer" containerID="98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4"
Jul 29 08:09:37.063364 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:09:37.063345    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4\": container with ID starting with 98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4 not found: ID does not exist" containerID="98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4"
Jul 29 08:09:37.063412 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.063374    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4"} err="failed to get container status \"98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4\": rpc error: code = NotFound desc = could not find container \"98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4\": container with ID starting with 98dfc163a827ae4d936bdda197b863975f9e1adc9b241fdfd14be04bb987c3b4 not found: ID does not exist"
Jul 29 08:09:37.075838 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.075812    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"]
Jul 29 08:09:37.079692 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:37.079669    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-940406-574d5864d-5qzlp"]
Jul 29 08:09:38.790108 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:38.790075    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd61c412-17b1-4c85-af7a-0d06829799a7" path="/var/lib/kubelet/pods/dd61c412-17b1-4c85-af7a-0d06829799a7/volumes"
Jul 29 08:09:41.779720 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:41.779680    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"]
Jul 29 08:09:45.664363 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:45.664329    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:09:56.798657 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:56.798616    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s" podUID="fc3c297e-d9f8-4769-b9e8-b3335472ffc3" containerName="main" containerID="cri-o://9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508" gracePeriod=45
Jul 29 08:09:57.033447 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.033421    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:09:57.146241 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.146201    2577 generic.go:356] "Generic (PLEG): container finished" podID="fc3c297e-d9f8-4769-b9e8-b3335472ffc3" containerID="9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508" exitCode=0
Jul 29 08:09:57.146419 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.146245    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"
Jul 29 08:09:57.146419 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.146278    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s" event={"ID":"fc3c297e-d9f8-4769-b9e8-b3335472ffc3","Type":"ContainerDied","Data":"9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508"}
Jul 29 08:09:57.146419 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.146325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s" event={"ID":"fc3c297e-d9f8-4769-b9e8-b3335472ffc3","Type":"ContainerDied","Data":"75d41bea801ad6f5483780d8e0bdc00a16ecfbd9341bc19e4ef150e9ee62c495"}
Jul 29 08:09:57.146419 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.146343    2577 scope.go:117] "RemoveContainer" containerID="9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508"
Jul 29 08:09:57.154647 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.154630    2577 scope.go:117] "RemoveContainer" containerID="9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508"
Jul 29 08:09:57.154928 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:09:57.154892    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508\": container with ID starting with 9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508 not found: ID does not exist" containerID="9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508"
Jul 29 08:09:57.155034 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.154936    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508"} err="failed to get container status \"9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508\": rpc error: code = NotFound desc = could not find container \"9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508\": container with ID starting with 9b499168b41ee507410821cae90e2b8b2ac4d29050fcf4c5394f8f17be371508 not found: ID does not exist"
Jul 29 08:09:57.197260 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197227    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tls-certs\") pod \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") "
Jul 29 08:09:57.197392 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-model-cache\") pod \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") "
Jul 29 08:09:57.197392 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197299    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-home\") pod \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") "
Jul 29 08:09:57.197392 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197338    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tmp-dir\") pod \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") "
Jul 29 08:09:57.197392 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197361    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-dshm\") pod \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") "
Jul 29 08:09:57.197392 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197390    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hndh8\" (UniqueName: \"kubernetes.io/projected/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-kube-api-access-hndh8\") pod \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\" (UID: \"fc3c297e-d9f8-4769-b9e8-b3335472ffc3\") "
Jul 29 08:09:57.197657 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197543    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "fc3c297e-d9f8-4769-b9e8-b3335472ffc3" (UID: "fc3c297e-d9f8-4769-b9e8-b3335472ffc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:09:57.197657 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197588    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-home" (OuterVolumeSpecName: "home") pod "fc3c297e-d9f8-4769-b9e8-b3335472ffc3" (UID: "fc3c297e-d9f8-4769-b9e8-b3335472ffc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:09:57.197764 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197743    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-model-cache\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:57.197764 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197746    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fc3c297e-d9f8-4769-b9e8-b3335472ffc3" (UID: "fc3c297e-d9f8-4769-b9e8-b3335472ffc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:09:57.197764 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.197761    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-home\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:57.199596 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.199569    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-dshm" (OuterVolumeSpecName: "dshm") pod "fc3c297e-d9f8-4769-b9e8-b3335472ffc3" (UID: "fc3c297e-d9f8-4769-b9e8-b3335472ffc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 08:09:57.199596 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.199583    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc3c297e-d9f8-4769-b9e8-b3335472ffc3" (UID: "fc3c297e-d9f8-4769-b9e8-b3335472ffc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:09:57.199745 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.199616    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-kube-api-access-hndh8" (OuterVolumeSpecName: "kube-api-access-hndh8") pod "fc3c297e-d9f8-4769-b9e8-b3335472ffc3" (UID: "fc3c297e-d9f8-4769-b9e8-b3335472ffc3"). InnerVolumeSpecName "kube-api-access-hndh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:09:57.298182 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.298151    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tls-certs\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:57.298182 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.298177    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-tmp-dir\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:57.298182 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.298185    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-dshm\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:57.298385 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.298193    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hndh8\" (UniqueName: \"kubernetes.io/projected/fc3c297e-d9f8-4769-b9e8-b3335472ffc3-kube-api-access-hndh8\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:09:57.468312 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.468278    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"]
Jul 29 08:09:57.472084 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:57.472059    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-940406/ipp-route-940406-kserve-dbd9b977f-pln8s"]
Jul 29 08:09:58.790505 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:09:58.790469    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc3c297e-d9f8-4769-b9e8-b3335472ffc3" path="/var/lib/kubelet/pods/fc3c297e-d9f8-4769-b9e8-b3335472ffc3/volumes"
Jul 29 08:10:28.347115 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347071    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"]
Jul 29 08:10:28.347651 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347593    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26beaa83-7bb5-4d89-a9c3-a02e336f5e42" containerName="revoke-keys"
Jul 29 08:10:28.347651 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347615    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="26beaa83-7bb5-4d89-a9c3-a02e336f5e42" containerName="revoke-keys"
Jul 29 08:10:28.347651 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347647    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd61c412-17b1-4c85-af7a-0d06829799a7" containerName="payload-pre-processing"
Jul 29 08:10:28.347820 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347656    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd61c412-17b1-4c85-af7a-0d06829799a7" containerName="payload-pre-processing"
Jul 29 08:10:28.347820 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347677    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc3c297e-d9f8-4769-b9e8-b3335472ffc3" containerName="main"
Jul 29 08:10:28.347820 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347686    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc3c297e-d9f8-4769-b9e8-b3335472ffc3" containerName="main"
Jul 29 08:10:28.347820 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347765    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="26beaa83-7bb5-4d89-a9c3-a02e336f5e42" containerName="revoke-keys"
Jul 29 08:10:28.347820 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347781    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc3c297e-d9f8-4769-b9e8-b3335472ffc3" containerName="main"
Jul 29 08:10:28.347820 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.347793    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd61c412-17b1-4c85-af7a-0d06829799a7" containerName="payload-pre-processing"
Jul 29 08:10:28.352434 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.352409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:28.355734 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.355713    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2z4n9\""
Jul 29 08:10:28.370639 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.370617    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"]
Jul 29 08:10:28.474123 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.474089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8389afb2-cf97-4478-9d8f-27e0c8a4844f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght\" (UID: \"8389afb2-cf97-4478-9d8f-27e0c8a4844f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:28.574919 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.574872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8389afb2-cf97-4478-9d8f-27e0c8a4844f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght\" (UID: \"8389afb2-cf97-4478-9d8f-27e0c8a4844f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:28.575518 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.575493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8389afb2-cf97-4478-9d8f-27e0c8a4844f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght\" (UID: \"8389afb2-cf97-4478-9d8f-27e0c8a4844f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:28.662520 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.662493    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:28.797951 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:28.797927    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"]
Jul 29 08:10:28.800525 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:10:28.800499    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8389afb2_cf97_4478_9d8f_27e0c8a4844f.slice/crio-8c93dcd7c3ab33f0503404a292c3bb0c4cd8e92ac95c54dfb7b02811f76f4910 WatchSource:0}: Error finding container 8c93dcd7c3ab33f0503404a292c3bb0c4cd8e92ac95c54dfb7b02811f76f4910: Status 404 returned error can't find the container with id 8c93dcd7c3ab33f0503404a292c3bb0c4cd8e92ac95c54dfb7b02811f76f4910
Jul 29 08:10:29.295265 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:29.295164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght" event={"ID":"8389afb2-cf97-4478-9d8f-27e0c8a4844f","Type":"ContainerStarted","Data":"d493f13b85b62812cfbf999d3236001bf1064608365fc8b95bf2501d44b49dd4"}
Jul 29 08:10:29.295265 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:29.295224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght" event={"ID":"8389afb2-cf97-4478-9d8f-27e0c8a4844f","Type":"ContainerStarted","Data":"8c93dcd7c3ab33f0503404a292c3bb0c4cd8e92ac95c54dfb7b02811f76f4910"}
Jul 29 08:10:29.345534 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:29.345482    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght" podStartSLOduration=1.34546425 podStartE2EDuration="1.34546425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 08:10:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:10:29.336192858 +0000 UTC m=+3311.084258700" watchObservedRunningTime="2026-07-29 08:10:29.34546425 +0000 UTC m=+3311.093530093"
Jul 29 08:10:30.300538 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:30.300501    2577 generic.go:356] "Generic (PLEG): container finished" podID="8389afb2-cf97-4478-9d8f-27e0c8a4844f" containerID="d493f13b85b62812cfbf999d3236001bf1064608365fc8b95bf2501d44b49dd4" exitCode=0
Jul 29 08:10:30.300976 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:30.300590    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght" event={"ID":"8389afb2-cf97-4478-9d8f-27e0c8a4844f","Type":"ContainerDied","Data":"d493f13b85b62812cfbf999d3236001bf1064608365fc8b95bf2501d44b49dd4"}
Jul 29 08:10:31.429840 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:31.429810    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:31.500678 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:31.500648    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8389afb2-cf97-4478-9d8f-27e0c8a4844f-maas-api-service-ca\") pod \"8389afb2-cf97-4478-9d8f-27e0c8a4844f\" (UID: \"8389afb2-cf97-4478-9d8f-27e0c8a4844f\") "
Jul 29 08:10:31.501043 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:31.501022    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8389afb2-cf97-4478-9d8f-27e0c8a4844f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8389afb2-cf97-4478-9d8f-27e0c8a4844f" (UID: "8389afb2-cf97-4478-9d8f-27e0c8a4844f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:10:31.602039 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:31.601959    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8389afb2-cf97-4478-9d8f-27e0c8a4844f-maas-api-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:10:32.310549 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:32.310521    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"
Jul 29 08:10:32.310549 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:32.310535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght" event={"ID":"8389afb2-cf97-4478-9d8f-27e0c8a4844f","Type":"ContainerDied","Data":"8c93dcd7c3ab33f0503404a292c3bb0c4cd8e92ac95c54dfb7b02811f76f4910"}
Jul 29 08:10:32.310769 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:32.310583    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c93dcd7c3ab33f0503404a292c3bb0c4cd8e92ac95c54dfb7b02811f76f4910"
Jul 29 08:10:35.397532 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.397496    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"]
Jul 29 08:10:35.397956 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.397692    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" podUID="e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" containerName="maas-api" containerID="cri-o://79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0" gracePeriod=30
Jul 29 08:10:35.663167 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.663142    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:10:35.741717 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.741681    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") pod \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") "
Jul 29 08:10:35.741887 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.741774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4rwz\" (UniqueName: \"kubernetes.io/projected/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-kube-api-access-c4rwz\") pod \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") "
Jul 29 08:10:35.741887 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.741816    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-openshift-service-ca\") pod \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\" (UID: \"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b\") "
Jul 29 08:10:35.742218 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.742194    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" (UID: "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:10:35.743882 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.743860    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" (UID: "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 08:10:35.743956 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.743936    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-kube-api-access-c4rwz" (OuterVolumeSpecName: "kube-api-access-c4rwz") pod "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" (UID: "e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b"). InnerVolumeSpecName "kube-api-access-c4rwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 08:10:35.829296 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.829255    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"]
Jul 29 08:10:35.836553 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.836524    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f1dc98-72ght"]
Jul 29 08:10:35.843291 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.843268    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4rwz\" (UniqueName: \"kubernetes.io/projected/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-kube-api-access-c4rwz\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:10:35.843412 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.843293    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-openshift-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:10:35.843412 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:35.843305    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b-maas-api-tls\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:10:36.329744 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.329705    2577 generic.go:356] "Generic (PLEG): container finished" podID="e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" containerID="79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0" exitCode=0
Jul 29 08:10:36.329953 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.329754    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"
Jul 29 08:10:36.329953 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.329790    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" event={"ID":"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b","Type":"ContainerDied","Data":"79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0"}
Jul 29 08:10:36.329953 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.329847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48" event={"ID":"e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b","Type":"ContainerDied","Data":"3151e1fb37622a6d09177539240a95e500c2bd02b20260f4351890092360e9a2"}
Jul 29 08:10:36.329953 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.329872    2577 scope.go:117] "RemoveContainer" containerID="79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0"
Jul 29 08:10:36.340923 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.340882    2577 scope.go:117] "RemoveContainer" containerID="79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0"
Jul 29 08:10:36.341183 ip-10-0-137-55 kubenswrapper[2577]: E0729 08:10:36.341164    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0\": container with ID starting with 79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0 not found: ID does not exist" containerID="79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0"
Jul 29 08:10:36.341259 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.341191    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0"} err="failed to get container status \"79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0\": rpc error: code = NotFound desc = could not find container \"79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0\": container with ID starting with 79fe8c3093c45b1321d11c61ea40acd471a067669b54b3216092282eca25a8e0 not found: ID does not exist"
Jul 29 08:10:36.372536 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.372511    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"]
Jul 29 08:10:36.397126 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.397098    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f1dc98-7d6674f6c7-92j48"]
Jul 29 08:10:36.789372 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.789340    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8389afb2-cf97-4478-9d8f-27e0c8a4844f" path="/var/lib/kubelet/pods/8389afb2-cf97-4478-9d8f-27e0c8a4844f/volumes"
Jul 29 08:10:36.789756 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:36.789707    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" path="/var/lib/kubelet/pods/e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b/volumes"
Jul 29 08:10:38.518426 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:38.518393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:10:42.971598 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971517    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"]
Jul 29 08:10:42.971988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971859    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" containerName="maas-api"
Jul 29 08:10:42.971988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971872    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" containerName="maas-api"
Jul 29 08:10:42.971988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971889    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8389afb2-cf97-4478-9d8f-27e0c8a4844f" containerName="revoke-keys"
Jul 29 08:10:42.971988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971895    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8389afb2-cf97-4478-9d8f-27e0c8a4844f" containerName="revoke-keys"
Jul 29 08:10:42.971988 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971985    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7ef5bcf-693c-4052-bb6e-6bfca5b53e9b" containerName="maas-api"
Jul 29 08:10:42.972153 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.971995    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8389afb2-cf97-4478-9d8f-27e0c8a4844f" containerName="revoke-keys"
Jul 29 08:10:42.976127 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.976107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:42.979518 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.979490    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2z4n9\""
Jul 29 08:10:42.979780 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.979760    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 08:10:42.985327 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:42.985303    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"]
Jul 29 08:10:43.112274 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:43.112240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f56314d-0ee4-4a62-9902-5a82252527c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-984886-bvjmp\" (UID: \"6f56314d-0ee4-4a62-9902-5a82252527c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:43.213206 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:43.213171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f56314d-0ee4-4a62-9902-5a82252527c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-984886-bvjmp\" (UID: \"6f56314d-0ee4-4a62-9902-5a82252527c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:43.213837 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:43.213809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f56314d-0ee4-4a62-9902-5a82252527c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-984886-bvjmp\" (UID: \"6f56314d-0ee4-4a62-9902-5a82252527c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:43.287053 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:43.286986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:43.418696 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:43.418670    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"]
Jul 29 08:10:43.420824 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:10:43.420798    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f56314d_0ee4_4a62_9902_5a82252527c3.slice/crio-2ed95de9988f469b149157fc80798d7106b0fb0ed1d3a11ab0974eb9cc38ba35 WatchSource:0}: Error finding container 2ed95de9988f469b149157fc80798d7106b0fb0ed1d3a11ab0974eb9cc38ba35: Status 404 returned error can't find the container with id 2ed95de9988f469b149157fc80798d7106b0fb0ed1d3a11ab0974eb9cc38ba35
Jul 29 08:10:44.373563 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:44.373464    2577 generic.go:356] "Generic (PLEG): container finished" podID="6f56314d-0ee4-4a62-9902-5a82252527c3" containerID="0ea173b9060ebe20b5f90b203baa2b0b6bbee08db7fd122ee1aec79cb9ab1b70" exitCode=0
Jul 29 08:10:44.373563 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:44.373514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp" event={"ID":"6f56314d-0ee4-4a62-9902-5a82252527c3","Type":"ContainerDied","Data":"0ea173b9060ebe20b5f90b203baa2b0b6bbee08db7fd122ee1aec79cb9ab1b70"}
Jul 29 08:10:44.374055 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:44.373589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp" event={"ID":"6f56314d-0ee4-4a62-9902-5a82252527c3","Type":"ContainerStarted","Data":"2ed95de9988f469b149157fc80798d7106b0fb0ed1d3a11ab0974eb9cc38ba35"}
Jul 29 08:10:45.507405 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:45.507379    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:45.633302 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:45.633202    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f56314d-0ee4-4a62-9902-5a82252527c3-maas-api-service-ca\") pod \"6f56314d-0ee4-4a62-9902-5a82252527c3\" (UID: \"6f56314d-0ee4-4a62-9902-5a82252527c3\") "
Jul 29 08:10:45.633590 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:45.633561    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f56314d-0ee4-4a62-9902-5a82252527c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f56314d-0ee4-4a62-9902-5a82252527c3" (UID: "6f56314d-0ee4-4a62-9902-5a82252527c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 08:10:45.733985 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:45.733958    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f56314d-0ee4-4a62-9902-5a82252527c3-maas-api-service-ca\") on node \"ip-10-0-137-55.ec2.internal\" DevicePath \"\""
Jul 29 08:10:46.384423 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:46.384390    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"
Jul 29 08:10:46.384600 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:46.384389    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp" event={"ID":"6f56314d-0ee4-4a62-9902-5a82252527c3","Type":"ContainerDied","Data":"2ed95de9988f469b149157fc80798d7106b0fb0ed1d3a11ab0974eb9cc38ba35"}
Jul 29 08:10:46.384600 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:46.384528    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ed95de9988f469b149157fc80798d7106b0fb0ed1d3a11ab0974eb9cc38ba35"
Jul 29 08:10:49.144970 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:49.144936    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"]
Jul 29 08:10:49.148192 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:49.148163    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-984886-bvjmp"]
Jul 29 08:10:50.789625 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:50.789586    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f56314d-0ee4-4a62-9902-5a82252527c3" path="/var/lib/kubelet/pods/6f56314d-0ee4-4a62-9902-5a82252527c3/volumes"
Jul 29 08:10:52.985580 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:10:52.985545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:11:05.083310 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:05.083271    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wvjfq"]
Jul 29 08:11:16.985951 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:16.985918    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qqdvk_4171bc0f-d2e1-4bff-a32f-29091b80541d/manager/0.log"
Jul 29 08:11:17.889878 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:17.889850    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6c966467-m8snl_230995ba-b9fb-48d8-b0cc-9345f8b60a3e/copy-manifests/0.log"
Jul 29 08:11:17.897505 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:17.897475    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6c966467-m8snl_230995ba-b9fb-48d8-b0cc-9345f8b60a3e/manager/0.log"
Jul 29 08:11:20.530276 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:20.530243    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cf4kx_97ed7bde-5e79-428a-abb7-c8fbe2a5ea15/kuadrant-console-plugin/0.log"
Jul 29 08:11:20.921089 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:20.921063    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wvjfq_6fb2d089-525f-410e-a6f2-972a5f1f2f8e/limitador/0.log"
Jul 29 08:11:21.786844 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:21.786816    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd888fd86-sn4q9_741a0e9a-3ee1-499b-ab50-65c1bc83c52c/kube-auth-proxy/0.log"
Jul 29 08:11:22.930164 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:22.930130    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-xgmzf_06373343-d7aa-48d8-a2f9-464b4fe182bc/main/0.log"
Jul 29 08:11:23.160776 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:23.160730    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rbq9t_68f91e4a-d089-4ce5-badf-dc1bef143add/main/0.log"
Jul 29 08:11:23.285579 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:23.285505    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-zz58q_fedebd96-4830-4bfd-8e96-ba4bc891f7ef/main/0.log"
Jul 29 08:11:32.061411 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:32.061377    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ntlft_0270a271-eec0-4a58-a6f1-a57996c15f95/global-pull-secret-syncer/0.log"
Jul 29 08:11:32.217565 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:32.217532    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c29rm_f37a9f39-8a38-4e4f-87c7-e835d1fb9f1b/konnectivity-agent/0.log"
Jul 29 08:11:32.345746 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:32.345670    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-55.ec2.internal_b0b909cb70da34197f82620fdf4d9c81/haproxy/0.log"
Jul 29 08:11:38.174438 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:38.174413    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-cf4kx_97ed7bde-5e79-428a-abb7-c8fbe2a5ea15/kuadrant-console-plugin/0.log"
Jul 29 08:11:38.554006 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:38.553895    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wvjfq_6fb2d089-525f-410e-a6f2-972a5f1f2f8e/limitador/0.log"
Jul 29 08:11:40.676438 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:40.676408    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c788db9b-hr9sb_78fd41e8-490a-4e1a-a3ae-1006fb0a6055/metrics-server/0.log"
Jul 29 08:11:40.707926 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:40.707873    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-pcbf6_a44532ec-4b0e-4b23-bd5f-05dd0c34157d/monitoring-plugin/0.log"
Jul 29 08:11:40.913322 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:40.913296    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xgq24_6f0604e2-1137-4c01-af59-10250bdb2ed6/node-exporter/0.log"
Jul 29 08:11:40.937387 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:40.937302    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xgq24_6f0604e2-1137-4c01-af59-10250bdb2ed6/kube-rbac-proxy/0.log"
Jul 29 08:11:40.959741 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:40.959720    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xgq24_6f0604e2-1137-4c01-af59-10250bdb2ed6/init-textfile/0.log"
Jul 29 08:11:40.985061 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:40.985034    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vrqj8_667d0ae0-7f9c-4014-a3dd-6f5f5273d920/kube-rbac-proxy-main/0.log"
Jul 29 08:11:41.007623 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.007603    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vrqj8_667d0ae0-7f9c-4014-a3dd-6f5f5273d920/kube-rbac-proxy-self/0.log"
Jul 29 08:11:41.031331 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.031304    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-vrqj8_667d0ae0-7f9c-4014-a3dd-6f5f5273d920/openshift-state-metrics/0.log"
Jul 29 08:11:41.076264 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.076234    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/prometheus/0.log"
Jul 29 08:11:41.098039 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.098010    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/config-reloader/0.log"
Jul 29 08:11:41.122176 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.122152    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/thanos-sidecar/0.log"
Jul 29 08:11:41.144095 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.144073    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/kube-rbac-proxy-web/0.log"
Jul 29 08:11:41.166633 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.166608    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/kube-rbac-proxy/0.log"
Jul 29 08:11:41.189052 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.189000    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/kube-rbac-proxy-thanos/0.log"
Jul 29 08:11:41.213242 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.213221    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fdc27892-e7ef-41a2-b3ea-4a96d0753927/init-config-reloader/0.log"
Jul 29 08:11:41.263150 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.263122    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h49x7_80a0c65c-352f-4abb-af52-a597cc6457df/prometheus-operator/0.log"
Jul 29 08:11:41.308606 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.308581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-h49x7_80a0c65c-352f-4abb-af52-a597cc6457df/kube-rbac-proxy/0.log"
Jul 29 08:11:41.335595 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:41.335567    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-nxqjx_caabdffe-32e1-4e3e-b216-e4abb7e24c50/prometheus-operator-admission-webhook/0.log"
Jul 29 08:11:43.176087 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.176053    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"]
Jul 29 08:11:43.176471 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.176378    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f56314d-0ee4-4a62-9902-5a82252527c3" containerName="revoke-keys"
Jul 29 08:11:43.176471 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.176390    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f56314d-0ee4-4a62-9902-5a82252527c3" containerName="revoke-keys"
Jul 29 08:11:43.176471 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.176438    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f56314d-0ee4-4a62-9902-5a82252527c3" containerName="revoke-keys"
Jul 29 08:11:43.179504 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.179487    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.189235 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.189213    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ztqbj\"/\"openshift-service-ca.crt\""
Jul 29 08:11:43.189337 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.189276    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ztqbj\"/\"kube-root-ca.crt\""
Jul 29 08:11:43.197415 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.197391    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ztqbj\"/\"default-dockercfg-jwhlp\""
Jul 29 08:11:43.222766 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.222726    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"]
Jul 29 08:11:43.238932 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.238887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-lib-modules\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.239055 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.238934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v2tm\" (UniqueName: \"kubernetes.io/projected/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-kube-api-access-2v2tm\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.239055 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.238975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-proc\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.239055 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.239022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-podres\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.239055 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.239045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-sys\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340001 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.339967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-lib-modules\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340001 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v2tm\" (UniqueName: \"kubernetes.io/projected/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-kube-api-access-2v2tm\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-proc\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-podres\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-sys\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340128    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-proc\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-sys\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340158    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-lib-modules\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.340190 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.340176    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-podres\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.351750 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.351721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v2tm\" (UniqueName: \"kubernetes.io/projected/2d7b2853-02cb-433d-a20b-1fae3d1ac66f-kube-api-access-2v2tm\") pod \"perf-node-gather-daemonset-hl9lh\" (UID: \"2d7b2853-02cb-433d-a20b-1fae3d1ac66f\") " pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.489579 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.489474    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:43.636184 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.636090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"]
Jul 29 08:11:43.639115 ip-10-0-137-55 kubenswrapper[2577]: W0729 08:11:43.639079    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d7b2853_02cb_433d_a20b_1fae3d1ac66f.slice/crio-3306ce34dac287295126d19213d2bfa897718422fe72f4c112afdf14ffdaf757 WatchSource:0}: Error finding container 3306ce34dac287295126d19213d2bfa897718422fe72f4c112afdf14ffdaf757: Status 404 returned error can't find the container with id 3306ce34dac287295126d19213d2bfa897718422fe72f4c112afdf14ffdaf757
Jul 29 08:11:43.647660 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:43.647630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh" event={"ID":"2d7b2853-02cb-433d-a20b-1fae3d1ac66f","Type":"ContainerStarted","Data":"3306ce34dac287295126d19213d2bfa897718422fe72f4c112afdf14ffdaf757"}
Jul 29 08:11:44.653813 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:44.653773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh" event={"ID":"2d7b2853-02cb-433d-a20b-1fae3d1ac66f","Type":"ContainerStarted","Data":"ed672fd3b1db013736da101a566bb08ff658f5d5ef4ccc623611c964ce4b97c5"}
Jul 29 08:11:44.654202 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:44.653849    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:44.684632 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:44.684577    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh" podStartSLOduration=1.684560059 podStartE2EDuration="1.684560059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 08:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 08:11:44.682346434 +0000 UTC m=+3386.430412277" watchObservedRunningTime="2026-07-29 08:11:44.684560059 +0000 UTC m=+3386.432625906"
Jul 29 08:11:46.137697 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:46.137670    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bdhzd_1a60808d-8e2c-4192-ae4e-7db2f04c3306/dns/0.log"
Jul 29 08:11:46.170407 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:46.170379    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bdhzd_1a60808d-8e2c-4192-ae4e-7db2f04c3306/kube-rbac-proxy/0.log"
Jul 29 08:11:46.271772 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:46.271746    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-825gr_92cfc2e5-14c0-4b60-a417-60a688d08203/dns-node-resolver/0.log"
Jul 29 08:11:47.039869 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:47.039828    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7x4dw_d9b974f2-1f07-4620-9199-4ff3c54c124f/node-ca/0.log"
Jul 29 08:11:48.738172 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:48.738142    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd888fd86-sn4q9_741a0e9a-3ee1-499b-ab50-65c1bc83c52c/kube-auth-proxy/0.log"
Jul 29 08:11:49.944736 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:49.944706    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cn4fx_58ecc536-8a5c-42d1-8180-31d417acfa9e/serve-healthcheck-canary/0.log"
Jul 29 08:11:50.573726 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:50.573695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b6nlg_2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0/kube-rbac-proxy/0.log"
Jul 29 08:11:50.602657 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:50.602629    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b6nlg_2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0/exporter/0.log"
Jul 29 08:11:50.630009 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:50.629979    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-b6nlg_2ab292ff-afd8-4b2e-b1eb-bfc61f5ff3e0/extractor/0.log"
Jul 29 08:11:50.669033 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:50.669009    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ztqbj/perf-node-gather-daemonset-hl9lh"
Jul 29 08:11:52.859607 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:52.859571    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd888fd86-sn4q9_741a0e9a-3ee1-499b-ab50-65c1bc83c52c/kube-auth-proxy/0.log"
Jul 29 08:11:53.495751 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:53.495705    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qqdvk_4171bc0f-d2e1-4bff-a32f-29091b80541d/manager/0.log"
Jul 29 08:11:54.465737 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:54.465707    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6c966467-m8snl_230995ba-b9fb-48d8-b0cc-9345f8b60a3e/manager/0.log"
Jul 29 08:11:54.503402 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:54.503376    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6c966467-m8snl_230995ba-b9fb-48d8-b0cc-9345f8b60a3e/copy-manifests/0.log"
Jul 29 08:11:54.771352 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:54.771321    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dd888fd86-sn4q9_741a0e9a-3ee1-499b-ab50-65c1bc83c52c/kube-auth-proxy/0.log"
Jul 29 08:11:55.949914 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:55.949864    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d85576cc8-zz7b6_33e73cf7-7ec4-484a-b1a4-36679b1e71da/manager/0.log"
Jul 29 08:11:55.979787 ip-10-0-137-55 kubenswrapper[2577]: I0729 08:11:55.979760    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4mskb_680a2962-a879-4443-96ed-009e74a88a29/openshift-lws-operator/0.log"