Jul 24 17:36:41.470649 ip-10-0-140-161 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 17:36:41.879054 ip-10-0-140-161 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:36:41.879054 ip-10-0-140-161 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 17:36:41.879054 ip-10-0-140-161 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:36:41.879054 ip-10-0-140-161 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 17:36:41.879054 ip-10-0-140-161 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 17:36:41.879802 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.879515    2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 17:36:41.884642 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884613    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:36:41.884642 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884641    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:36:41.884642 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884645    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:36:41.884642 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884648    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884651    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884654    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884657    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884659    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884662    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884665    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884668    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884670    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884673    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884676    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884683    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884686    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884688    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884691    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884694    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884696    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884699    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884702    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884704    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:36:41.884773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884707    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884709    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884712    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884714    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884717    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884719    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884722    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884725    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884727    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884730    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884733    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884735    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884738    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884741    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884744    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884748    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884750    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884754    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884757    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:36:41.885251 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884759    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884762    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884765    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884768    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884772    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884775    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884778    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884780    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884785    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884790    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884793    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884796    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884799    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884803    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884806    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884808    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884811    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884816    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884819    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:36:41.885747 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884822    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884825    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884828    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884830    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884833    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884836    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884839    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884842    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884846    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884848    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884851    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884854    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884857    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884860    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884864    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884866    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884869    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884872    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884875    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884880    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:36:41.886211 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884884    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884887    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884890    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884893    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884896    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884898    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884901    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884904    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884906    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884909    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884912    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884915    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884917    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884920    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884922    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884925    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884927    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.884930    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885691    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885700    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:36:41.886722 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885703    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885705    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885709    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885712    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885714    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885717    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885720    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885722    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885725    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885728    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885730    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885734    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885737    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885740    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885742    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885745    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885748    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885750    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885753    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:36:41.887229 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885756    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885759    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885762    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885764    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885767    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885770    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885772    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885775    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885778    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885780    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885783    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885786    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885790    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885793    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885795    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885798    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885801    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885803    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885806    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885808    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:36:41.887719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885811    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885813    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885816    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885818    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885821    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885824    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885830    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885834    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885837    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885840    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885843    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885846    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885850    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885854    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885858    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885861    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885865    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885868    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885871    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:36:41.888230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885873    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885876    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885879    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885882    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885885    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885888    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885892    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885894    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885898    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885901    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885904    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885907    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885910    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885912    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885915    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885917    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885921    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885924    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885927    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885930    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:36:41.888726 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885932    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885935    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885937    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885940    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885943    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885945    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885948    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885950    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885953    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885956    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885959    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885961    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885963    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885966    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885969    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885972    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885974    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885977    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.885980    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886417    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:36:41.889208 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886423    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886426    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886429    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886432    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886435    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886438    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886443    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886446    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886449    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886452    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886454    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886460    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886464    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886466    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886471    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886475    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886478    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886481    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886484    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886487    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:36:41.889715 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886490    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886493    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886495    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886498    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886500    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886503    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886505    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886509    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886513    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886516    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886523    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886526    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886528    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886531    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886534    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886536    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886540    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886542    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886546    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886549    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:36:41.890221 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886552    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886554    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886557    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886560    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886564    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886566    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886569    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886571    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886575    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886578    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886580    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886583    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886585    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886588    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886590    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886592    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886595    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886598    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886600    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:36:41.890719 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886603    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886605    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886608    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886611    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886613    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886616    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886618    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886621    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886623    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886626    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886629    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886634    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886636    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886639    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886641    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886644    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886646    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886649    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886652    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:36:41.891174 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886654    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886657    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886659    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886662    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886664    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886667    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886670    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886672    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886675    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886677    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886680    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886682    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886685    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886687    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886689    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886692    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886695    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886697    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886700    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.886703    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:36:41.891725 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886795    2575 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886803    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886809    2575 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886815    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886830    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886834    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886838    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886843    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886846    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886849    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886853    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886857    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886860    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886864    2575 flags.go:64] FLAG: --cgroup-root=""
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886867    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886870    2575 flags.go:64] FLAG: --client-ca-file=""
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886872    2575 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886876    2575 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886881    2575 flags.go:64] FLAG: --cluster-domain=""
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886884    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886887    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886890    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886894    2575 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886900    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 17:36:41.892219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886904    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886907    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886911    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886914    2575 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886917    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886920    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886924    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886926    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886931    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886934    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886937    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886940    2575 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886943    2575 flags.go:64] FLAG: --enable-server="true"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886946    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886951    2575 flags.go:64] FLAG: --event-burst="100"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886955    2575 flags.go:64] FLAG: --event-qps="50"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886958    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886961    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886964    2575 flags.go:64] FLAG: --eviction-hard=""
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886968    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886971    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886975    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886978    2575 flags.go:64] FLAG: --eviction-soft=""
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886982    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886985    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 17:36:41.892820 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886988    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886991    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886994    2575 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886997    2575 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.886999    2575 flags.go:64] FLAG: --feature-gates=""
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887003    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887008    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887011    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887014    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887017    2575 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887020    2575 flags.go:64] FLAG: --help="false"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887023    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-140-161.ec2.internal"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887026    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887030    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887033    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887037    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887040    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887043    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887046    2575 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887049    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887052    2575 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887055    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887058    2575 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887061    2575 flags.go:64] FLAG: --kube-reserved=""
Jul 24 17:36:41.893499 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887064    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887067    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887070    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887073    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887076    2575 flags.go:64] FLAG: --lock-file=""
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887082    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887085    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887088    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887093    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887097    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887099    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887103    2575 flags.go:64] FLAG: --logging-format="text"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887106    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887109    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887113    2575 flags.go:64] FLAG: --manifest-url=""
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887116    2575 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887120    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887123    2575 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887127    2575 flags.go:64] FLAG: --max-pods="110"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887131    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887134    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887137    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887139    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887143    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887146    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 17:36:41.894091 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887148    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887157    2575 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887160    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887163    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887166    2575 flags.go:64] FLAG: --pod-cidr=""
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887169    2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887176    2575 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887179    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887183    2575 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887186    2575 flags.go:64] FLAG: --port="10250"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887189    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887192    2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-036af10550a3a31b2"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887195    2575 flags.go:64] FLAG: --qos-reserved=""
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887199    2575 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887202    2575 flags.go:64] FLAG: --register-node="true"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887205    2575 flags.go:64] FLAG: --register-with-taints=""
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887210    2575 flags.go:64] FLAG: --registry-burst="10"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887212    2575 flags.go:64] FLAG: --registry-qps="5"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887215    2575 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887218    2575 flags.go:64] FLAG: --reserved-memory=""
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887222    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887225    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887230    2575 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887233    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887236    2575 flags.go:64] FLAG: --runonce="false"
Jul 24 17:36:41.894712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887240    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887243    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887246    2575 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887249    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887252    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887255    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887259    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887274    2575 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887279    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887282    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887285    2575 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887288    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887291    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887295    2575 flags.go:64] FLAG: --system-cgroups=""
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887298    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887303    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887306    2575 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887309    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887314    2575 flags.go:64] FLAG: --tls-min-version=""
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887317    2575 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887320    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887324    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887327    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887330    2575 flags.go:64] FLAG: --v="2"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887334    2575 flags.go:64] FLAG: --version="false"
Jul 24 17:36:41.895425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887339    2575 flags.go:64] FLAG: --vmodule=""
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887342    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.887346    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887467    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887473    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887476    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887480    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887483    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887486    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887489    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887491    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887494    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887497    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887499    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887502    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887506    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887509    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887513    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887517    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:36:41.896047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887520    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887523    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887525    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887528    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887531    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887533    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887536    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887539    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887541    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887544    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887547    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887550    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887552    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887555    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887557    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887560    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887564    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887569    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887572    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887575    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:36:41.896516 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887577    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887580    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887583    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887586    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887589    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887591    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887594    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887597    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887601    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887604    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887606    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887609    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887611    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887614    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887617    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887619    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887622    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887624    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887627    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887630    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:36:41.897047 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887632    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887635    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887638    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887641    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887644    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887647    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887649    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887651    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887654    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887659    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887661    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887664    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887667    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887669    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887672    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887675    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887677    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887680    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887683    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:36:41.897591 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887686    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887692    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887695    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887699    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887701    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887704    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887706    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887709    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887712    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887714    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887717    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887720    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887723    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887726    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887729    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887731    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887734    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887737    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887740    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887742    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:36:41.898058 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887745    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887747    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887751    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.887754    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.888200    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.895055    2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.895074    2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.895091    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.895100    2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895160    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895166    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895169    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895172    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895176    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895179    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:36:41.898569 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895181    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895184    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895186    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895189    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895192    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895194    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895197    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895200    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895202    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895205    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895208    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895210    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895213    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895215    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895219    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895221    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895224    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895226    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895229    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895232    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:36:41.898947 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895235    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895237    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895240    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895244    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895247    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895250    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895252    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895255    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895257    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895260    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895274    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895277    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895282    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895285    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895289    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895292    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895295    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895298    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895301    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:36:41.899439 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895304    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895306    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895309    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895312    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895314    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895317    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895319    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895322    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895325    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895328    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895331    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895334    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895337    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895340    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895342    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895345    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895349    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895352    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895355    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:36:41.899920 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895357    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895360    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895363    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895366    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895369    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895371    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895374    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895376    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895380    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895384    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895387    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895389    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895393    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895395    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895398    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895401    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895404    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895406    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895409    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:36:41.900450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895412    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895414    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895417    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895420    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895423    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895426    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895430    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895434    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895438    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895441    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895444    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895447    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895450    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895452    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895455    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:36:41.900917 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895457    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.895463    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895590    2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895595    2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895598    2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895601    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895605    2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895608    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895610    2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895613    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895616    2575 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895618    2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895621    2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895624    2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895627    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 17:36:41.901386 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895629    2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895632    2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895635    2575 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895637    2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895640    2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895643    2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895646    2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895649    2575 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895651    2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895654    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895656    2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895659    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895661    2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895664    2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895666    2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895669    2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895671    2575 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895676    2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895679    2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 17:36:41.901759 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895683    2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895687    2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895690    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895693    2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895696    2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895699    2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895702    2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895705    2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895708    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895710    2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895713    2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895715    2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895717    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895720    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895723    2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895726    2575 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895729    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895731    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895734    2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895737    2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 17:36:41.902224 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895740    2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895742    2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895745    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895747    2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895750    2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895753    2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895756    2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895758    2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895761    2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895764    2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895766    2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895769    2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895771    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895774    2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895777    2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895779    2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895782    2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895784    2575 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895787    2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895790    2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 17:36:41.902721 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895792    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895794    2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895797    2575 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895799    2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895802    2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895804    2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895807    2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895809    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895812    2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895815    2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895818    2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895821    2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895824    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895830    2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895834    2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895837    2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895839    2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895843    2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895846    2575 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 17:36:41.903201 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895848    2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895851    2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895854    2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895857    2575 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895860    2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895862    2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895867    2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:41.895870    2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.895875    2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.896020    2575 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.898112    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.899461    2575 server.go:1019] "Starting client certificate rotation"
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.899563    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:36:41.903669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.899633    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 17:36:41.904002 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.902213    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:36:41.905726 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.905705    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 17:36:41.915825 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.915802    2575 log.go:25] "Validated CRI v1 runtime API"
Jul 24 17:36:41.917931 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.917913    2575 log.go:25] "Validated CRI v1 image API"
Jul 24 17:36:41.918830 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.918812    2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 17:36:41.923404 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.923377    2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8e6851c7-e782-43bd-b453-27a9eaf65bf5:/dev/nvme0n1p3 e391f32b-482a-44ac-80f0-042b3f3b58c5:/dev/nvme0n1p4]
Jul 24 17:36:41.923502 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.923402    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 17:36:41.929036 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.928879    2575 manager.go:216] Machine: {Timestamp:2026-07-24 17:36:41.927673645 +0000 UTC m=+0.331614864 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102143 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26fbda039fe5240068b065c162506e SystemUUID:ec26fbda-039f-e524-0068-b065c162506e BootID:27e5fc29-b986-4dae-ae31-9303f3ed0921 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2a:fe:c2:7c:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2a:fe:c2:7c:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:7a:cc:d2:d3:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 17:36:41.929036 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.929022    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 17:36:41.929174 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.929121    2575 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 17:36:41.929362 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.929344    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:36:41.932275 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.932227    2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 17:36:41.932452 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.932286    2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 17:36:41.933508 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.933495    2575 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 17:36:41.933572 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.933515    2575 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 17:36:41.933572 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.933527    2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:36:41.934322 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.934310    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 17:36:41.935112 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.935100    2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 17:36:41.936850 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.936835    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:36:41.937056 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937043    2575 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 17:36:41.937123 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937085    2575 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 17:36:41.937123 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937103    2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 17:36:41.937202 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937124    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 17:36:41.937202 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937138    2575 kubelet.go:398] "Adding apiserver pod source"
Jul 24 17:36:41.937202 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937152    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 17:36:41.937576 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937562    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:36:41.937642 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.937584    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 17:36:41.939503 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.939484    2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 17:36:41.941276 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.941253    2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 17:36:41.941325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.941298    2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 17:36:41.943035 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943022    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 17:36:41.943084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943040    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 17:36:41.943084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943047    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 17:36:41.943084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943052    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 17:36:41.943084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943063    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 17:36:41.943084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943072    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 17:36:41.943084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943083    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 17:36:41.943249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943089    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 17:36:41.943249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943095    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 17:36:41.943249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943101    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 17:36:41.943249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943110    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 17:36:41.943249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.943115    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 17:36:41.944036 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.944027    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 17:36:41.944036 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.944036    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 17:36:41.947492 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.947474    2575 server.go:1268] "Started kubelet"
Jul 24 17:36:41.947618 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.947557    2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 17:36:41.947680 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.947623    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 17:36:41.947729 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.947699    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 17:36:41.948333 ip-10-0-140-161 systemd[1]: Started Kubernetes Kubelet.
Jul 24 17:36:41.950944 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.950909    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 17:36:41.951130 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.951111    2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 17:36:41.951204 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.951178    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-95x9g"
Jul 24 17:36:41.952690 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.951745    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 17:36:41.952690 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.951904    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 17:36:41.952868 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.952850    2575 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 17:36:41.954994 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.954968    2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 17:36:41.957371 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.956462    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-161.ec2.internal.18c54a5fa37c2d8d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-161.ec2.internal,UID:ip-10-0-140-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-161.ec2.internal,},FirstTimestamp:2026-07-24 17:36:41.947442573 +0000 UTC m=+0.351383792,LastTimestamp:2026-07-24 17:36:41.947442573 +0000 UTC m=+0.351383792,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-161.ec2.internal,}"
Jul 24 17:36:41.957470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.957453    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-95x9g"
Jul 24 17:36:41.960463 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.960443    2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 17:36:41.960581 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.960452    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 17:36:41.960769 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.960745    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:41.962805 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.962783    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 17:36:41.962908 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.962825    2575 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 17:36:41.962908 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.962842    2575 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 17:36:41.963023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.962951    2575 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 17:36:41.963023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.962958    2575 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 17:36:41.963099 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963036    2575 factory.go:153] Registering CRI-O factory
Jul 24 17:36:41.963099 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963055    2575 factory.go:223] Registration of the crio container factory successfully
Jul 24 17:36:41.963173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963112    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 17:36:41.963173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963123    2575 factory.go:55] Registering systemd factory
Jul 24 17:36:41.963173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963131    2575 factory.go:223] Registration of the systemd container factory successfully
Jul 24 17:36:41.963173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963154    2575 factory.go:103] Registering Raw factory
Jul 24 17:36:41.963173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963167    2575 manager.go:1196] Started watching for new ooms in manager
Jul 24 17:36:41.963925 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.963906    2575 manager.go:349] Starting recovery of all containers
Jul 24 17:36:41.966537 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.966502    2575 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 17:36:41.966676 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:41.966628    2575 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 17:36:41.973704 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.973683    2575 manager.go:354] Recovery completed
Jul 24 17:36:41.977685 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.977665    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:41.979758 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.979739    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:41.979871 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.979773    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:41.979871 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.979784    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:41.980308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.980292    2575 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 17:36:41.980366 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.980310    2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 17:36:41.980366 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.980329    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:41.980366 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.980334    2575 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 17:36:41.982199 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.982176    2575 policy_none.go:49] "None policy: Start"
Jul 24 17:36:41.982317 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.982204    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 17:36:41.982317 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.982220    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 17:36:41.982762 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.982740    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:41.982841 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.982807    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:41.982841 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.982823    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:41.983520 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:41.983505    2575 policy_none.go:47] "Start"
Jul 24 17:36:42.023132 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.022952    2575 manager.go:337] "Starting Device Plugin manager"
Jul 24 17:36:42.023279 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.023163    2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 17:36:42.023279 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.023178    2575 server.go:87] "Starting device plugin registration server"
Jul 24 17:36:42.023466 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.023453    2575 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 17:36:42.023529 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.023468    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 17:36:42.023579 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.023565    2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 17:36:42.023668 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.023651    2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 17:36:42.023668 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.023664    2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 17:36:42.024299 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.024253    2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 17:36:42.024383 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.024336    2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.124520 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.124489    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:42.126359 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.126090    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:42.126359 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.126124    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:42.126359 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.126146    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:42.126359 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.126197    2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.129800 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.129727    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 17:36:42.131046 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.131024    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 17:36:42.131150 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.131063    2575 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 17:36:42.131150 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.131100    2575 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 17:36:42.131150 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.131146    2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 17:36:42.132048 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.132031    2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.132130 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.132059    2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-161.ec2.internal\": node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.133157 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.133140    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:36:42.199695 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.199665    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.231749 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.231714    2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"]
Jul 24 17:36:42.231847 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.231819    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:42.233380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.233363    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:42.233465 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.233397    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:42.233465 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.233409    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:42.235481 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.235468    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:42.235591 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.235578    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.235628 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.235608    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:42.236363 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.236330    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:42.236363 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.236354    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:42.236503 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.236374    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:42.236503 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.236375    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:42.236503 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.236385    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:42.236503 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.236393    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:42.238515 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.238500    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.238567 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.238525    2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 17:36:42.239275 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.239245    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 17:36:42.239365 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.239287    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 17:36:42.239365 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.239297    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeHasSufficientPID"
Jul 24 17:36:42.264023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.263988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dfdc9f251e260e601debe29346b8d757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal\" (UID: \"dfdc9f251e260e601debe29346b8d757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.264023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.264022    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfdc9f251e260e601debe29346b8d757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal\" (UID: \"dfdc9f251e260e601debe29346b8d757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.264224 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.264040    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/42c822a09c30d0b2448c94638a327500-config\") pod \"kube-apiserver-proxy-ip-10-0-140-161.ec2.internal\" (UID: \"42c822a09c30d0b2448c94638a327500\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.266171 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.266148    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-161.ec2.internal\" not found" node="ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.270805 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.270783    2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-161.ec2.internal\" not found" node="ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.300085 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.300043    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.364925 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.364889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfdc9f251e260e601debe29346b8d757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal\" (UID: \"dfdc9f251e260e601debe29346b8d757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.365056 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.364926    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/42c822a09c30d0b2448c94638a327500-config\") pod \"kube-apiserver-proxy-ip-10-0-140-161.ec2.internal\" (UID: \"42c822a09c30d0b2448c94638a327500\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.365056 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.364955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dfdc9f251e260e601debe29346b8d757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal\" (UID: \"dfdc9f251e260e601debe29346b8d757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.365056 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.364996    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dfdc9f251e260e601debe29346b8d757-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal\" (UID: \"dfdc9f251e260e601debe29346b8d757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.365056 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.365023    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/42c822a09c30d0b2448c94638a327500-config\") pod \"kube-apiserver-proxy-ip-10-0-140-161.ec2.internal\" (UID: \"42c822a09c30d0b2448c94638a327500\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.365056 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.364998    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dfdc9f251e260e601debe29346b8d757-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal\" (UID: \"dfdc9f251e260e601debe29346b8d757\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.401116 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.401040    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.501993 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.501967    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.568614 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.568574    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.573691 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.573664    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:42.603014 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.602973    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.703703 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.703614    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.804244 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.804204    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.861221 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.861188    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:36:42.899278 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.899239    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 17:36:42.899808 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.899430    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:36:42.899808 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.899449    2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 17:36:42.904565 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:42.904537    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:42.966493 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.966385    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 17:31:42 +0000 UTC" deadline="2028-03-06 10:29:10.651067663 +0000 UTC"
Jul 24 17:36:42.966493 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:42.966414    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14176h52m27.684656441s"
Jul 24 17:36:43.005635 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:43.005599    2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-161.ec2.internal\" not found"
Jul 24 17:36:43.101636 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.101605    2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:36:43.161084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.161049    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:43.173460 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.173433    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:36:43.175747 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.175729    2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal"
Jul 24 17:36:43.189331 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.189303    2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 17:36:43.192314 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:43.192281    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c822a09c30d0b2448c94638a327500.slice/crio-e08e93bf82a05cc6f1dda06a55261df9ae06dac56263a5810f66b599f5e59e25 WatchSource:0}: Error finding container e08e93bf82a05cc6f1dda06a55261df9ae06dac56263a5810f66b599f5e59e25: Status 404 returned error can't find the container with id e08e93bf82a05cc6f1dda06a55261df9ae06dac56263a5810f66b599f5e59e25
Jul 24 17:36:43.192773 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:43.192751    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfdc9f251e260e601debe29346b8d757.slice/crio-7d5e0f11ee46e208fdbbe3996522b5ca7d0b667c9ac088dbea8fb5a5d7def43c WatchSource:0}: Error finding container 7d5e0f11ee46e208fdbbe3996522b5ca7d0b667c9ac088dbea8fb5a5d7def43c: Status 404 returned error can't find the container with id 7d5e0f11ee46e208fdbbe3996522b5ca7d0b667c9ac088dbea8fb5a5d7def43c
Jul 24 17:36:43.196813 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.196797    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:36:43.553324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.553293    2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:36:43.938547 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.938456    2575 apiserver.go:52] "Watching apiserver"
Jul 24 17:36:43.944711 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.944681    2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 17:36:43.945219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.945178    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b","openshift-cluster-node-tuning-operator/tuned-plwzj","openshift-dns/node-resolver-59zmp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal","openshift-multus/multus-2dnl4","openshift-network-diagnostics/network-check-target-24w52","openshift-network-operator/iptables-alerter-hssks","openshift-ovn-kubernetes/ovnkube-node-92zjc","kube-system/global-pull-secret-syncer-57pv5","kube-system/konnectivity-agent-ksf6l","kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal","openshift-image-registry/node-ca-gtbnj","openshift-multus/multus-additional-cni-plugins-zdmf6","openshift-multus/network-metrics-daemon-x2cmc"]
Jul 24 17:36:43.947938 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.947906    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:43.950427 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.950365    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 17:36:43.950552 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.950507    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.950851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.950832    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 17:36:43.951223 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.951037    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pkxb6\""
Jul 24 17:36:43.952550 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.952531    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 17:36:43.953228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.952761    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 17:36:43.953228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.952826    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x7qwt\""
Jul 24 17:36:43.953228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.953050    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 17:36:43.953228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.953068    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 17:36:43.953502 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.953368    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.953980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.953958    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:43.955904 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.955883    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 17:36:43.956099 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.956072    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dcwnb\""
Jul 24 17:36:43.956221 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.956119    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.956735 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.956437    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:43.956735 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:43.956512    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:43.958617 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.958594    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.961089 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.960844    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 17:36:43.961089 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.960932    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nm6jb\""
Jul 24 17:36:43.961089 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.960945    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 17:36:43.961331 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.961201    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:43.961331 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:43.961249    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:43.963601 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.963582    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:43.963708 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:43.963652    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:43.966079 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.966054    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:43.969400 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.968969    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c7fpk\""
Jul 24 17:36:43.969400 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.969246    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.969400 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.969303    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:36:43.969624 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.969548    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 17:36:43.971580 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.971555    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:43.974082 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.974045    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:36:43.974340 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.974317    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.974420 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.974388    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dvz5p\""
Jul 24 17:36:43.974779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.974757    2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 17:36:43.974877 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.974857    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:43.975879 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.975854    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-conf-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.975988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.975892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6b88495-8247-4300-88ad-215fe6cadd63-multus-daemon-config\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.975988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.975918    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-socket-dir-parent\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.975988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.975946    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjsh\" (UniqueName: \"kubernetes.io/projected/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-kube-api-access-dsjsh\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:43.975988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.975970    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-system-cni-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976211 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976068    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-os-release\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976211 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976102    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-netns\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976211 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976128    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-cni-multus\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976211 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976161    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b88495-8247-4300-88ad-215fe6cadd63-cni-binary-copy\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976211 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976189    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cni-binary-copy\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976226    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-tmp-dir\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976257    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-cnibin\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976300    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-cni-bin\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976365    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-system-cni-dir\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-os-release\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976423    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49qcq\" (UniqueName: \"kubernetes.io/projected/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-kube-api-access-49qcq\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:43.976491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976483    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-kubelet\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76828591-503d-4d58-87cb-a570db5fc73f-kubelet-config\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-hosts-file\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976566    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-k8s-cni-cncf-io\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976590    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-hostroot\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-etc-kubernetes\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976666    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fde4c405-d38a-4578-ba07-de3466e88cb0-konnectivity-ca\") pod \"konnectivity-agent-ksf6l\" (UID: \"fde4c405-d38a-4578-ba07-de3466e88cb0\") " pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976692    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-cni-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-multus-certs\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84665\" (UniqueName: \"kubernetes.io/projected/b6b88495-8247-4300-88ad-215fe6cadd63-kube-api-access-84665\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fde4c405-d38a-4578-ba07-de3466e88cb0-agent-certs\") pod \"konnectivity-agent-ksf6l\" (UID: \"fde4c405-d38a-4578-ba07-de3466e88cb0\") " pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976796    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cnibin\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976823    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbm6\" (UniqueName: \"kubernetes.io/projected/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-kube-api-access-ddbm6\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976874    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:43.976974 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976896    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76828591-503d-4d58-87cb-a570db5fc73f-dbus\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:43.977866 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976922    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:43.977866 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.976971    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 17:36:43.977866 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.977209    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 17:36:43.977866 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.977282    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.977866 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.977340    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-trs78\""
Jul 24 17:36:43.977866 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.977680    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:43.980131 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980080    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 17:36:43.980131 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980107    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 17:36:43.980324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980159    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.980324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980261    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 17:36:43.980324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980284    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:43.980486 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980431    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rzdqp\""
Jul 24 17:36:43.980540 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980523    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 17:36:43.980750 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.980729    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 17:36:43.982557 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.982534    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v6t7j\""
Jul 24 17:36:43.983118 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.982950    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 17:36:43.983118 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.982972    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 17:36:43.983118 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:43.982954    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 17:36:44.002108 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.002079    2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgwfc"
Jul 24 17:36:44.011681 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.011644    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgwfc"
Jul 24 17:36:44.064824 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.064784    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 17:36:44.077235 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-etc-kubernetes\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.077235 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fde4c405-d38a-4578-ba07-de3466e88cb0-konnectivity-ca\") pod \"konnectivity-agent-ksf6l\" (UID: \"fde4c405-d38a-4578-ba07-de3466e88cb0\") " pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysctl-d\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077320    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-host\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077347    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grbd4\" (UniqueName: \"kubernetes.io/projected/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-kube-api-access-grbd4\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077371    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-etc-selinux\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077396    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-cni-netd\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077422    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-env-overrides\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-cni-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077474    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-multus-certs\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077498    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84665\" (UniqueName: \"kubernetes.io/projected/b6b88495-8247-4300-88ad-215fe6cadd63-kube-api-access-84665\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.077523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077521    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fde4c405-d38a-4578-ba07-de3466e88cb0-agent-certs\") pod \"konnectivity-agent-ksf6l\" (UID: \"fde4c405-d38a-4578-ba07-de3466e88cb0\") " pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077547    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cnibin\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077591    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lht9s\" (UniqueName: \"kubernetes.io/projected/0186ebba-bc46-4681-8c76-3e52e1f2467f-kube-api-access-lht9s\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077614    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-etc-kubernetes\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077617    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovn-node-metrics-cert\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077683    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76828591-503d-4d58-87cb-a570db5fc73f-dbus\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077736    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-run\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077763    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-var-lib-kubelet\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077790    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88x5l\" (UniqueName: \"kubernetes.io/projected/44d6f106-a431-47d0-bc2b-00d5a656d522-kube-api-access-88x5l\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077814    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-systemd\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077865    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jx97\" (UniqueName: \"kubernetes.io/projected/31bf0864-6453-4621-b0f0-6ee5864b52c7-kube-api-access-9jx97\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077891    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0186ebba-bc46-4681-8c76-3e52e1f2467f-iptables-alerter-script\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.077961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-conf-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-socket-dir-parent\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078045 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078015    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-multus-certs\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078033    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-socket-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-slash\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078078    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-cni-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b88495-8247-4300-88ad-215fe6cadd63-cni-binary-copy\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078090    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cnibin\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078077    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76828591-503d-4d58-87cb-a570db5fc73f-dbus\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.078197    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078221    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-conf-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078254    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fde4c405-d38a-4578-ba07-de3466e88cb0-konnectivity-ca\") pod \"konnectivity-agent-ksf6l\" (UID: \"fde4c405-d38a-4578-ba07-de3466e88cb0\") " pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.078288    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:44.578245204 +0000 UTC m=+2.982186432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078308    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cni-binary-copy\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078301    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-multus-socket-dir-parent\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078388    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-kubernetes\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078384    2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078511    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysctl-conf\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078542    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-kubelet\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.078971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078572    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-cni-bin\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078618    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-system-cni-dir\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-os-release\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0186ebba-bc46-4681-8c76-3e52e1f2467f-host-slash\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078711    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b88495-8247-4300-88ad-215fe6cadd63-cni-binary-copy\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-cni-bin\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078783    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-system-cni-dir\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078859    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-os-release\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078883    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49qcq\" (UniqueName: \"kubernetes.io/projected/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-kube-api-access-49qcq\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078899    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cni-binary-copy\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078915    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-kubelet\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.078952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-var-lib-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079032    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovnkube-config\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-kubelet\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079123    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-hosts-file\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079157    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-systemd\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079180    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-tmp\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.079897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-hosts-file\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079235    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-log-socket\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079299    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovnkube-script-lib\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-k8s-cni-cncf-io\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079385    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-hostroot\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079413    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079463    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-k8s-cni-cncf-io\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqmbw\" (UniqueName: \"kubernetes.io/projected/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-kube-api-access-vqmbw\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079526    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-hostroot\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079527    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-node-log\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.079529    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.079607    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:36:44.579588928 +0000 UTC m=+2.983530146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079641    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbm6\" (UniqueName: \"kubernetes.io/projected/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-kube-api-access-ddbm6\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079687    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.080851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079728    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-serviceca\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-modprobe-d\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079821    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-sys\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079850    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6b88495-8247-4300-88ad-215fe6cadd63-multus-daemon-config\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjsh\" (UniqueName: \"kubernetes.io/projected/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-kube-api-access-dsjsh\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079900    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-system-cni-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079924    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-os-release\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079949    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-netns\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079976    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-host\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.079999    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysconfig\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080021    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-lib-modules\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-cni-multus\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080069    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-sys-fs\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080114    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-run-netns\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080135    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-system-cni-dir\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080255    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-os-release\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.081701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-run-netns\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080138    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-etc-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-tmp-dir\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-cnibin\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-host-var-lib-cni-multus\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080417    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080430    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b88495-8247-4300-88ad-215fe6cadd63-cnibin\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080442    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080446    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080497    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-device-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080543    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b6b88495-8247-4300-88ad-215fe6cadd63-multus-daemon-config\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080572    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080600    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-tuned\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080627    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080739    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-registration-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080764    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-ovn\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.082522 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-cni-bin\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.083415 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-tmp-dir\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.083415 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080823    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76828591-503d-4d58-87cb-a570db5fc73f-kubelet-config\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:44.083415 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080915    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-systemd-units\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.083415 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.080932    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76828591-503d-4d58-87cb-a570db5fc73f-kubelet-config\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:44.084188 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.084160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fde4c405-d38a-4578-ba07-de3466e88cb0-agent-certs\") pod \"konnectivity-agent-ksf6l\" (UID: \"fde4c405-d38a-4578-ba07-de3466e88cb0\") " pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:44.087468 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.087439    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:36:44.087468 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.087468    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:36:44.087682 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.087483    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:44.087682 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.087558    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:44.587535629 +0000 UTC m=+2.991476855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:44.088729 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.088667    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84665\" (UniqueName: \"kubernetes.io/projected/b6b88495-8247-4300-88ad-215fe6cadd63-kube-api-access-84665\") pod \"multus-2dnl4\" (UID: \"b6b88495-8247-4300-88ad-215fe6cadd63\") " pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.089649 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.089625    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbm6\" (UniqueName: \"kubernetes.io/projected/78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58-kube-api-access-ddbm6\") pod \"multus-additional-cni-plugins-zdmf6\" (UID: \"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58\") " pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.089818 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.089799    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49qcq\" (UniqueName: \"kubernetes.io/projected/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-kube-api-access-49qcq\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:44.090642 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.090618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjsh\" (UniqueName: \"kubernetes.io/projected/3855ca0f-6f2c-4fbc-b018-b5b0033ae11d-kube-api-access-dsjsh\") pod \"node-resolver-59zmp\" (UID: \"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d\") " pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.136500 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.136433    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal" event={"ID":"42c822a09c30d0b2448c94638a327500","Type":"ContainerStarted","Data":"e08e93bf82a05cc6f1dda06a55261df9ae06dac56263a5810f66b599f5e59e25"}
Jul 24 17:36:44.137933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.137902    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal" event={"ID":"dfdc9f251e260e601debe29346b8d757","Type":"ContainerStarted","Data":"7d5e0f11ee46e208fdbbe3996522b5ca7d0b667c9ac088dbea8fb5a5d7def43c"}
Jul 24 17:36:44.149400 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.149367    2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:36:44.182153 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182113    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-serviceca\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.182153 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182160    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-modprobe-d\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182189    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-sys\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-host\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysconfig\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182273    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-lib-modules\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-sys-fs\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182318    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-sys\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182338    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-run-netns\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182362    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-etc-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182358    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-modprobe-d\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182394    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysconfig\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182428 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182411    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-device-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182454    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-host\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182471    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-device-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182499    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-run-netns\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182521    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-etc-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182530    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-lib-modules\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182555    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-sys-fs\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182566    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-tuned\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-run-ovn-kubernetes\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182625    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182654    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-registration-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-ovn\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182715    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-cni-bin\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182725    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-systemd-units\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysctl-d\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.182980 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-host\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182818    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-registration-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182728    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-serviceca\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182830    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grbd4\" (UniqueName: \"kubernetes.io/projected/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-kube-api-access-grbd4\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182875    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-etc-selinux\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182900    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-cni-netd\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-env-overrides\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182952    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lht9s\" (UniqueName: \"kubernetes.io/projected/0186ebba-bc46-4681-8c76-3e52e1f2467f-kube-api-access-lht9s\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182977    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovn-node-metrics-cert\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-run\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-var-lib-kubelet\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysctl-d\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183083    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88x5l\" (UniqueName: \"kubernetes.io/projected/44d6f106-a431-47d0-bc2b-00d5a656d522-kube-api-access-88x5l\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-systemd\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183132    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-host\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183180    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-systemd-units\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.183910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jx97\" (UniqueName: \"kubernetes.io/projected/31bf0864-6453-4621-b0f0-6ee5864b52c7-kube-api-access-9jx97\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0186ebba-bc46-4681-8c76-3e52e1f2467f-iptables-alerter-script\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183222    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-cni-bin\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183240    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-socket-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183264    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-slash\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183280    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-var-lib-kubelet\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183288    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-kubernetes\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183333    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysctl-conf\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183358    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-kubelet\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183390    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0186ebba-bc46-4681-8c76-3e52e1f2467f-host-slash\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183426    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-var-lib-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovnkube-config\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-systemd\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183500    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-tmp\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183525    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-log-socket\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183528    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-env-overrides\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183551    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovnkube-script-lib\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183593    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqmbw\" (UniqueName: \"kubernetes.io/projected/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-kube-api-access-vqmbw\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183614    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-etc-selinux\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-node-log\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-cni-netd\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183718    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183764    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-kubelet\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183765    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-sysctl-conf\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183815    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0186ebba-bc46-4681-8c76-3e52e1f2467f-host-slash\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183857    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-var-lib-openvswitch\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183930    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-log-socket\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.183988    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-systemd\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184014    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-systemd\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.182779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-run-ovn\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184366    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/44d6f106-a431-47d0-bc2b-00d5a656d522-socket-dir\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184435    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-run\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184480    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-kubernetes\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184517    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.185798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184520    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-node-log\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184578    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31bf0864-6453-4621-b0f0-6ee5864b52c7-host-slash\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.184691    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0186ebba-bc46-4681-8c76-3e52e1f2467f-iptables-alerter-script\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.185052    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovnkube-config\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.185169    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovnkube-script-lib\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.185512    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-etc-tuned\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.186103    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/31bf0864-6453-4621-b0f0-6ee5864b52c7-ovn-node-metrics-cert\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.186779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.186113    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-tmp\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.198021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.197069    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grbd4\" (UniqueName: \"kubernetes.io/projected/fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4-kube-api-access-grbd4\") pod \"tuned-plwzj\" (UID: \"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4\") " pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.198021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.197086    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jx97\" (UniqueName: \"kubernetes.io/projected/31bf0864-6453-4621-b0f0-6ee5864b52c7-kube-api-access-9jx97\") pod \"ovnkube-node-92zjc\" (UID: \"31bf0864-6453-4621-b0f0-6ee5864b52c7\") " pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.198021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.197112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lht9s\" (UniqueName: \"kubernetes.io/projected/0186ebba-bc46-4681-8c76-3e52e1f2467f-kube-api-access-lht9s\") pod \"iptables-alerter-hssks\" (UID: \"0186ebba-bc46-4681-8c76-3e52e1f2467f\") " pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.198964 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.198941    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqmbw\" (UniqueName: \"kubernetes.io/projected/fe83c62b-750b-4ae3-a4ae-5ca6a056c125-kube-api-access-vqmbw\") pod \"node-ca-gtbnj\" (UID: \"fe83c62b-750b-4ae3-a4ae-5ca6a056c125\") " pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.199785 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.199759    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88x5l\" (UniqueName: \"kubernetes.io/projected/44d6f106-a431-47d0-bc2b-00d5a656d522-kube-api-access-88x5l\") pod \"aws-ebs-csi-driver-node-gfn7b\" (UID: \"44d6f106-a431-47d0-bc2b-00d5a656d522\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.262573 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.262519    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:36:44.272493 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.272448    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zdmf6"
Jul 24 17:36:44.282512 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.282488    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-59zmp"
Jul 24 17:36:44.290655 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.290626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2dnl4"
Jul 24 17:36:44.300623 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.300597    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hssks"
Jul 24 17:36:44.308627 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.308598    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-plwzj"
Jul 24 17:36:44.317494 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.317468    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b"
Jul 24 17:36:44.332636 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.332603    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:36:44.339659 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.339630    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gtbnj"
Jul 24 17:36:44.587508 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.587414    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:44.587508 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.587506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:44.587784 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.587582    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:44.587784 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.587591    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:44.587784 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.587654    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:45.587638262 +0000 UTC m=+3.991579488 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:44.587784 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.587669    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:36:45.587663261 +0000 UTC m=+3.991604483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:44.688184 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:44.688132    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:44.688402 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.688345    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:36:44.688402 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.688371    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:36:44.688402 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.688392    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:44.688570 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:44.688451    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:45.688435598 +0000 UTC m=+4.092376808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:45.015410 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:45.015287    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:31:44 +0000 UTC" deadline="2028-04-16 16:47:00.06839072 +0000 UTC"
Jul 24 17:36:45.015410 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:45.015359    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15167h10m15.053036346s"
Jul 24 17:36:45.131571 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:45.131527    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:45.131775 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.131685    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:45.201811 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.201778    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6b88495_8247_4300_88ad_215fe6cadd63.slice/crio-17eebb73aec475ae02e92da691d2f010f17be01cdfd1626eb5adb0eb4241a280 WatchSource:0}: Error finding container 17eebb73aec475ae02e92da691d2f010f17be01cdfd1626eb5adb0eb4241a280: Status 404 returned error can't find the container with id 17eebb73aec475ae02e92da691d2f010f17be01cdfd1626eb5adb0eb4241a280
Jul 24 17:36:45.224278 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.224249    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde4c405_d38a_4578_ba07_de3466e88cb0.slice/crio-876141922b9ce48831e93b1495ef2c4bfe5511cbab69a7ab6a1720f954ebe7da WatchSource:0}: Error finding container 876141922b9ce48831e93b1495ef2c4bfe5511cbab69a7ab6a1720f954ebe7da: Status 404 returned error can't find the container with id 876141922b9ce48831e93b1495ef2c4bfe5511cbab69a7ab6a1720f954ebe7da
Jul 24 17:36:45.225965 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.225829    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe83c62b_750b_4ae3_a4ae_5ca6a056c125.slice/crio-d441da3762be23bc2c981e8e750594349ef9628d37849a0cdf9205413271e6e4 WatchSource:0}: Error finding container d441da3762be23bc2c981e8e750594349ef9628d37849a0cdf9205413271e6e4: Status 404 returned error can't find the container with id d441da3762be23bc2c981e8e750594349ef9628d37849a0cdf9205413271e6e4
Jul 24 17:36:45.228385 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.228358    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31bf0864_6453_4621_b0f0_6ee5864b52c7.slice/crio-68501bd4f340322217fc299c717ed4964dbd24629f31276e9b68dac4ac507b7b WatchSource:0}: Error finding container 68501bd4f340322217fc299c717ed4964dbd24629f31276e9b68dac4ac507b7b: Status 404 returned error can't find the container with id 68501bd4f340322217fc299c717ed4964dbd24629f31276e9b68dac4ac507b7b
Jul 24 17:36:45.229450 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.229427    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf6ea1c_9b29_4516_a3bf_d20a1964d1f4.slice/crio-e5ef6a723487c38da93b5fab15e0f42a4c4750be1a0d2b34b2597aacdc673e20 WatchSource:0}: Error finding container e5ef6a723487c38da93b5fab15e0f42a4c4750be1a0d2b34b2597aacdc673e20: Status 404 returned error can't find the container with id e5ef6a723487c38da93b5fab15e0f42a4c4750be1a0d2b34b2597aacdc673e20
Jul 24 17:36:45.230764 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.230742    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0186ebba_bc46_4681_8c76_3e52e1f2467f.slice/crio-b793bc74bcb785c78833cf6bdac4b2fa253004de6ddc38da42a528be7e10b084 WatchSource:0}: Error finding container b793bc74bcb785c78833cf6bdac4b2fa253004de6ddc38da42a528be7e10b084: Status 404 returned error can't find the container with id b793bc74bcb785c78833cf6bdac4b2fa253004de6ddc38da42a528be7e10b084
Jul 24 17:36:45.232230 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.232191    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78ca0b7a_0d9d_4bff_82fb_6bbc80ad0b58.slice/crio-94c2276232ce134db6cbeb9288386e841a71d0948e790a5de5361b520ae01d29 WatchSource:0}: Error finding container 94c2276232ce134db6cbeb9288386e841a71d0948e790a5de5361b520ae01d29: Status 404 returned error can't find the container with id 94c2276232ce134db6cbeb9288386e841a71d0948e790a5de5361b520ae01d29
Jul 24 17:36:45.233863 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.232992    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d6f106_a431_47d0_bc2b_00d5a656d522.slice/crio-a90fe5845e9b50b32a1fe6b6730513824de60a0f9b51e58aa8f8f7ed1654169c WatchSource:0}: Error finding container a90fe5845e9b50b32a1fe6b6730513824de60a0f9b51e58aa8f8f7ed1654169c: Status 404 returned error can't find the container with id a90fe5845e9b50b32a1fe6b6730513824de60a0f9b51e58aa8f8f7ed1654169c
Jul 24 17:36:45.235839 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:36:45.235811    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3855ca0f_6f2c_4fbc_b018_b5b0033ae11d.slice/crio-88f7146f5ff08c1db2df4704b280173f681337cfa1992171a93fa775e0ab6608 WatchSource:0}: Error finding container 88f7146f5ff08c1db2df4704b280173f681337cfa1992171a93fa775e0ab6608: Status 404 returned error can't find the container with id 88f7146f5ff08c1db2df4704b280173f681337cfa1992171a93fa775e0ab6608
Jul 24 17:36:45.596197 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:45.595821    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:45.596197 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.595986    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:45.596460 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:45.596210    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:45.596460 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.596262    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:36:47.596240783 +0000 UTC m=+6.000182015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:45.596460 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.596314    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:45.596460 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.596367    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:47.596354946 +0000 UTC m=+6.000296158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:45.697441 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:45.696669    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:45.697441 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.696897    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:36:45.697441 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.696925    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:36:45.697441 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.696939    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:45.697441 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:45.697013    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:47.69698865 +0000 UTC m=+6.100929864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:46.015987 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.015851    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 17:31:44 +0000 UTC" deadline="2028-02-02 11:31:21.819026819 +0000 UTC"
Jul 24 17:36:46.015987 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.015949    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13385h54m35.803082757s"
Jul 24 17:36:46.050030 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.049779    2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 17:36:46.134519 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.133788    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:46.134519 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:46.133924    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:46.134519 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.134330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:46.134519 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:46.134446    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:46.161898 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.161855    2575 generic.go:356] "Generic (PLEG): container finished" podID="dfdc9f251e260e601debe29346b8d757" containerID="64d3893e106e38072f8d8647e35bb7b99dbe219e9e1b6897a115a59a886d61ba" exitCode=0
Jul 24 17:36:46.162249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.162207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal" event={"ID":"dfdc9f251e260e601debe29346b8d757","Type":"ContainerDied","Data":"64d3893e106e38072f8d8647e35bb7b99dbe219e9e1b6897a115a59a886d61ba"}
Jul 24 17:36:46.171578 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.171544    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerStarted","Data":"94c2276232ce134db6cbeb9288386e841a71d0948e790a5de5361b520ae01d29"}
Jul 24 17:36:46.180841 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.180714    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-plwzj" event={"ID":"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4","Type":"ContainerStarted","Data":"e5ef6a723487c38da93b5fab15e0f42a4c4750be1a0d2b34b2597aacdc673e20"}
Jul 24 17:36:46.184697 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.184618    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ksf6l" event={"ID":"fde4c405-d38a-4578-ba07-de3466e88cb0","Type":"ContainerStarted","Data":"876141922b9ce48831e93b1495ef2c4bfe5511cbab69a7ab6a1720f954ebe7da"}
Jul 24 17:36:46.190715 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.190635    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2dnl4" event={"ID":"b6b88495-8247-4300-88ad-215fe6cadd63","Type":"ContainerStarted","Data":"17eebb73aec475ae02e92da691d2f010f17be01cdfd1626eb5adb0eb4241a280"}
Jul 24 17:36:46.199305 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.199214    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal" event={"ID":"42c822a09c30d0b2448c94638a327500","Type":"ContainerStarted","Data":"29ed30432ce64a55fb81747fcf5a9010e3f8ef34c66ad782f11e450143ce357a"}
Jul 24 17:36:46.215092 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.214825    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-59zmp" event={"ID":"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d","Type":"ContainerStarted","Data":"88f7146f5ff08c1db2df4704b280173f681337cfa1992171a93fa775e0ab6608"}
Jul 24 17:36:46.220011 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.219924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b" event={"ID":"44d6f106-a431-47d0-bc2b-00d5a656d522","Type":"ContainerStarted","Data":"a90fe5845e9b50b32a1fe6b6730513824de60a0f9b51e58aa8f8f7ed1654169c"}
Jul 24 17:36:46.226473 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.226179    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hssks" event={"ID":"0186ebba-bc46-4681-8c76-3e52e1f2467f","Type":"ContainerStarted","Data":"b793bc74bcb785c78833cf6bdac4b2fa253004de6ddc38da42a528be7e10b084"}
Jul 24 17:36:46.232232 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.232157    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-161.ec2.internal" podStartSLOduration=2.231883891 podStartE2EDuration="2.231883891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:36:46.227326297 +0000 UTC m=+4.631267530" watchObservedRunningTime="2026-07-24 17:36:46.231883891 +0000 UTC m=+4.635825125"
Jul 24 17:36:46.240217 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.240063    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"68501bd4f340322217fc299c717ed4964dbd24629f31276e9b68dac4ac507b7b"}
Jul 24 17:36:46.259308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:46.259264    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gtbnj" event={"ID":"fe83c62b-750b-4ae3-a4ae-5ca6a056c125","Type":"ContainerStarted","Data":"d441da3762be23bc2c981e8e750594349ef9628d37849a0cdf9205413271e6e4"}
Jul 24 17:36:47.131611 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:47.131573    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:47.132119 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.131741    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:47.301731 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:47.301418    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal" event={"ID":"dfdc9f251e260e601debe29346b8d757","Type":"ContainerStarted","Data":"b11e3cee570c0c7942c4300aa763000e4c59531f0b8b4906e594fd0a4cf63737"}
Jul 24 17:36:47.319178 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:47.319109    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-161.ec2.internal" podStartSLOduration=3.319083187 podStartE2EDuration="3.319083187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:36:47.318756859 +0000 UTC m=+5.722698092" watchObservedRunningTime="2026-07-24 17:36:47.319083187 +0000 UTC m=+5.723024421"
Jul 24 17:36:47.618539 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:47.618447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:47.618539 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:47.618528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:47.618796 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.618702    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:47.618796 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.618769    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:36:51.618748519 +0000 UTC m=+10.022689734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:47.618914 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.618839    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:47.618914 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.618877    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:51.618863912 +0000 UTC m=+10.022805122 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:47.719297 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:47.719255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:47.719703 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.719540    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:36:47.719703 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.719564    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:36:47.719703 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.719578    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:47.719703 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:47.719645    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:51.719625284 +0000 UTC m=+10.123566499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:48.134033 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:48.133964    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:48.134549 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:48.134124    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:48.134549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:48.134211    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:48.134549 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:48.134290    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:49.132757 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:49.132714    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:49.132960 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:49.132912    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:50.131659 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:50.131419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:50.131659 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:50.131419    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:50.131659 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:50.131586    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:50.131659 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:50.131642    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:51.132156 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:51.132112    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:51.132632 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.132283    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:51.655574 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:51.655531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:51.655788 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:51.655632    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:51.655788 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.655771    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:51.655926 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.655840    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:59.655818175 +0000 UTC m=+18.059759407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:51.656281 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.656262    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:51.656456 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.656323    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:36:59.656304462 +0000 UTC m=+18.060245686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:51.756597 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:51.756540    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:51.756824 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.756755    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:36:51.756928 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.756824    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:36:51.756928 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.756839    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:51.756928 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:51.756918    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:36:59.756895248 +0000 UTC m=+18.160836479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:52.133008 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:52.132892    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:52.133500 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:52.133035    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:52.133500 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:52.133106    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:52.133500 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:52.133214    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:53.131585 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:53.131552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:53.131808 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:53.131695    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:54.132240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:54.132203    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:54.132773 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:54.132383    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:54.135471 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:54.132156    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:54.135620 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:54.135594    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:55.132231 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:55.132187    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:55.132458 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:55.132386    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:56.132292 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:56.132247    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:56.132481 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:56.132311    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:56.132481 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:56.132416    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:56.132939 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:56.132510    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:57.132138 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:57.132052    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:57.132307 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:57.132182    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:58.131603 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:58.131536    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:58.132023 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:58.131658    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:36:58.132023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:58.131698    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:58.132023 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:58.131778    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:36:59.131608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:59.131566    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:59.132105 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.131722    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:36:59.722343 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:59.722301    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:36:59.722645 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:59.722374    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:36:59.722645 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.722466    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:59.722645 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.722486    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:59.722645 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.722546    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:15.722527304 +0000 UTC m=+34.126468529 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:36:59.722645 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.722561    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:37:15.722555185 +0000 UTC m=+34.126496392 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:36:59.823236 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:36:59.823199    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:36:59.823440 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.823416    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:36:59.823490 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.823449    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:36:59.823490 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.823462    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:36:59.823572 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:36:59.823532    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:15.823513664 +0000 UTC m=+34.227454874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:37:00.132370 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:00.132282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:00.132823 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:00.132282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:00.132823 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:00.132418    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:00.132823 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:00.132504    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:01.132147 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:01.132096    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:01.132355 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:01.132252    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:02.132424 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:02.132376    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:02.132908 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:02.132485    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:02.132908 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:02.132542    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:02.132908 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:02.132657    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:03.131852 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:03.131819    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:03.132047 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:03.131949    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:04.132395 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:04.132356    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:04.132856 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:04.132359    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:04.132856 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:04.132492    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:04.132856 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:04.132573    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:05.132187 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.131821    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:05.132409 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:05.132320    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:05.345941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.345845    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b" event={"ID":"44d6f106-a431-47d0-bc2b-00d5a656d522","Type":"ContainerStarted","Data":"5768a3ba2a2d53d98c8e0a779f3a2510ac93fcab86cf7e1d24f53442a3b0978d"}
Jul 24 17:37:05.349444 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.349411    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"f4d6b40392af5be150435126ba2d3d7567edd0c856638b158825a875086430ad"}
Jul 24 17:37:05.349604 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.349497    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"a97b53d8383c64c01c92b4f49b12f0de6af98122a9cdeea52bb988462be1ac4e"}
Jul 24 17:37:05.349604 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.349544    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"c2e93eea4a906c342e883c65405e875c8f62ebc0e5ad709d87c87d82b0a336b8"}
Jul 24 17:37:05.349604 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.349590    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"b3492f75ac333d76b6855ecdd3c0b2c202b032ac3ccd9bed383cdd7204ca1936"}
Jul 24 17:37:05.349764 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.349637    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"62cd4862e25b3dd2402897b111c02316c869d94c6402e81815ccc5f83c78d59b"}
Jul 24 17:37:05.349764 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.349687    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"83b99a889ff943f0967a583e14269bc5be7dca695a305855248ac4b787868dcd"}
Jul 24 17:37:05.351017 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.350981    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gtbnj" event={"ID":"fe83c62b-750b-4ae3-a4ae-5ca6a056c125","Type":"ContainerStarted","Data":"397dd64d8981a27bf1a6df6026db951542e1995732195554fb01d98806f85f1e"}
Jul 24 17:37:05.352720 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.352697    2575 generic.go:356] "Generic (PLEG): container finished" podID="78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58" containerID="ab36f2885231bae46cce6fc82a2d7f29628e9d1aaa0ed54d32267ebdb9bc118d" exitCode=0
Jul 24 17:37:05.352829 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.352779    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerDied","Data":"ab36f2885231bae46cce6fc82a2d7f29628e9d1aaa0ed54d32267ebdb9bc118d"}
Jul 24 17:37:05.355075 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.354913    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-plwzj" event={"ID":"fcf6ea1c-9b29-4516-a3bf-d20a1964d1f4","Type":"ContainerStarted","Data":"e622516fc6ae51d6bd953a31244431494c37fa5911d95cf1249ea80734fe1e63"}
Jul 24 17:37:05.356649 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.356623    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ksf6l" event={"ID":"fde4c405-d38a-4578-ba07-de3466e88cb0","Type":"ContainerStarted","Data":"e12498f6e98a709df2fc62390bf6d49d184f63f7cf9b377b85cc3d7e75ecaccd"}
Jul 24 17:37:05.358228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.358188    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2dnl4" event={"ID":"b6b88495-8247-4300-88ad-215fe6cadd63","Type":"ContainerStarted","Data":"c34f1cf821a64624fe4116dcaba473d93d0fc6f0d0d78b36cf842de6c86e0da4"}
Jul 24 17:37:05.358394 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.358380    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:37:05.359798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.359773    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-59zmp" event={"ID":"3855ca0f-6f2c-4fbc-b018-b5b0033ae11d","Type":"ContainerStarted","Data":"18d01e32e82c021bb5b2f8aef4392908b7e3e24d6290e138d646c23025bd585b"}
Jul 24 17:37:05.373347 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.373287    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gtbnj" podStartSLOduration=3.195600597 podStartE2EDuration="22.37325797s" totalImagesPullingTime="19.177657373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:05.372623669 +0000 UTC m=+23.776564931" watchObservedRunningTime="2026-07-24 17:37:05.37325797 +0000 UTC m=+23.777199199"
Jul 24 17:37:05.399542 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.399491    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-59zmp" podStartSLOduration=3.22671936 podStartE2EDuration="22.399474398s" totalImagesPullingTime="19.172755038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:05.399175693 +0000 UTC m=+23.803116924" watchObservedRunningTime="2026-07-24 17:37:05.399474398 +0000 UTC m=+23.803415627"
Jul 24 17:37:05.429699 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.429640    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2dnl4" podStartSLOduration=3.176605808 podStartE2EDuration="22.429622557s" totalImagesPullingTime="19.253016749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:05.428606109 +0000 UTC m=+23.832547370" watchObservedRunningTime="2026-07-24 17:37:05.429622557 +0000 UTC m=+23.833563786"
Jul 24 17:37:05.452038 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.451985    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-plwzj" podStartSLOduration=3.255671292 podStartE2EDuration="22.451959955s" totalImagesPullingTime="19.196288663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:05.451838829 +0000 UTC m=+23.855780049" watchObservedRunningTime="2026-07-24 17:37:05.451959955 +0000 UTC m=+23.855901183"
Jul 24 17:37:05.480457 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.480396    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ksf6l" podStartSLOduration=3.297218787 podStartE2EDuration="22.480380956s" totalImagesPullingTime="19.183162169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:05.480002815 +0000 UTC m=+23.883944044" watchObservedRunningTime="2026-07-24 17:37:05.480380956 +0000 UTC m=+23.884322185"
Jul 24 17:37:05.938080 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:05.938050    2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 17:37:06.036297 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.036160    2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T17:37:05.938072222Z","UUID":"3fa5df35-fecf-4d63-ad30-6dfb8983c58a","Handler":null,"Name":"","Endpoint":""}
Jul 24 17:37:06.037661 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.037638    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 17:37:06.037780 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.037680    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 17:37:06.131995 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.131904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:06.132147 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:06.132042    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:06.132147 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.132089    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:06.132249 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:06.132226    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:06.364029 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.363987    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b" event={"ID":"44d6f106-a431-47d0-bc2b-00d5a656d522","Type":"ContainerStarted","Data":"c6d7aba4f35b04197045e154fabb5cb6bd8fa5c72bab0b50822d75ca40794e44"}
Jul 24 17:37:06.365726 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.365654    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hssks" event={"ID":"0186ebba-bc46-4681-8c76-3e52e1f2467f","Type":"ContainerStarted","Data":"ae77e466b544c0492911d125c616ed5294e1982747ac9e0b6f8fa4f9b701c7c1"}
Jul 24 17:37:06.381169 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:06.381106    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hssks" podStartSLOduration=4.201460211 podStartE2EDuration="23.381082723s" totalImagesPullingTime="19.179622512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:06.380865354 +0000 UTC m=+24.784806583" watchObservedRunningTime="2026-07-24 17:37:06.381082723 +0000 UTC m=+24.785023953"
Jul 24 17:37:07.131561 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:07.131520    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:07.131730 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:07.131693    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:07.370835 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:07.370800    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b" event={"ID":"44d6f106-a431-47d0-bc2b-00d5a656d522","Type":"ContainerStarted","Data":"64abc408d56ff4fd0e4baf942b2e15b46ce1e7138d9a225b33edda56a5245730"}
Jul 24 17:37:07.374770 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:07.374736    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"05c51ad62f44de59ff05ae6de0852235aadeb608cba9c0d7805dc8c21cc63425"}
Jul 24 17:37:07.395297 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:07.393745    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gfn7b" podStartSLOduration=3.006038744 podStartE2EDuration="24.393723371s" totalImagesPullingTime="21.387684627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:07.390903963 +0000 UTC m=+25.794845192" watchObservedRunningTime="2026-07-24 17:37:07.393723371 +0000 UTC m=+25.797664600"
Jul 24 17:37:08.131861 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:08.131821    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:08.131861 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:08.131862    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:08.132104 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:08.131963    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:08.132104 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:08.132037    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:08.880850 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:08.880578    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:37:08.881337 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:08.881232    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:37:09.131652 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:09.131564    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:09.131822 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:09.131704    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:09.380344 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:09.380315    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ksf6l"
Jul 24 17:37:10.131918 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.131737    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:10.132637 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.131782    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:10.132637 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:10.131994    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:10.132637 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:10.132078    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:10.384179 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.384052    2575 generic.go:356] "Generic (PLEG): container finished" podID="78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58" containerID="7d2219b1ef137c4f10803579a5286c859c5f77abe3dc10b23b7d5dac2c75c8aa" exitCode=0
Jul 24 17:37:10.384179 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.384142    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerDied","Data":"7d2219b1ef137c4f10803579a5286c859c5f77abe3dc10b23b7d5dac2c75c8aa"}
Jul 24 17:37:10.387816 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.387786    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" event={"ID":"31bf0864-6453-4621-b0f0-6ee5864b52c7","Type":"ContainerStarted","Data":"09e1939fe65da4031e5f9555b4fad165469b9673ba42360b8e6cddbb09270088"}
Jul 24 17:37:10.388226 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.388167    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:37:10.388383 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.388236    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:37:10.388383 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.388301    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:37:10.404003 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.403975    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:37:10.404130 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:10.404092    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:37:11.132222 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:11.132183    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:11.132770 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:11.132351    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:12.056098 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.055995    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc" podStartSLOduration=9.805066405 podStartE2EDuration="29.055977592s" totalImagesPullingTime="19.250911187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:10.441802495 +0000 UTC m=+28.845743726" watchObservedRunningTime="2026-07-24 17:37:12.055977592 +0000 UTC m=+30.459918821"
Jul 24 17:37:12.056509 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.056490    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-57pv5"]
Jul 24 17:37:12.056617 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.056584    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:12.056685 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:12.056660    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:12.059459 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.059430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-24w52"]
Jul 24 17:37:12.059577 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.059536    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:12.059642 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:12.059624    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:12.060483 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.060459    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x2cmc"]
Jul 24 17:37:12.060583 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.060556    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:12.060681 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:12.060658    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:12.394646 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.394541    2575 generic.go:356] "Generic (PLEG): container finished" podID="78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58" containerID="53e86355d6c36c1c963d1ffe8b236cee824ff8aae862e33b26136c8ad6dc0bd2" exitCode=0
Jul 24 17:37:12.394646 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:12.394621    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerDied","Data":"53e86355d6c36c1c963d1ffe8b236cee824ff8aae862e33b26136c8ad6dc0bd2"}
Jul 24 17:37:13.131516 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:13.131482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:13.131702 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:13.131623    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:14.131669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:14.131438    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:14.132131 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:14.131438    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:14.132131 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:14.131706    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:14.132131 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:14.131821    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:14.402049 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:14.401963    2575 generic.go:356] "Generic (PLEG): container finished" podID="78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58" containerID="ffd7e01aaf3acf3135ddf1da1b88a0167ed85bbae8c4ded62691ac65fcce73cb" exitCode=0
Jul 24 17:37:14.402049 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:14.402003    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerDied","Data":"ffd7e01aaf3acf3135ddf1da1b88a0167ed85bbae8c4ded62691ac65fcce73cb"}
Jul 24 17:37:15.131418 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:15.131380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:15.131591 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.131514    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:15.735089 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:15.735052    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:15.735089 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:15.735097    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:15.735598 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.735218    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 17:37:15.735598 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.735303    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret podName:76828591-503d-4d58-87cb-a570db5fc73f nodeName:}" failed. No retries permitted until 2026-07-24 17:37:47.735286308 +0000 UTC m=+66.139227527 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret") pod "global-pull-secret-syncer-57pv5" (UID: "76828591-503d-4d58-87cb-a570db5fc73f") : object "kube-system"/"original-pull-secret" not registered
Jul 24 17:37:15.735598 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.735218    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:37:15.735598 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.735408    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:47.735387161 +0000 UTC m=+66.139328385 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 17:37:15.836385 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:15.836350    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:15.836589 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.836492    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 17:37:15.836589 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.836516    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 17:37:15.836589 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.836528    2575 projected.go:196] Error preparing data for projected volume kube-api-access-pgrjg for pod openshift-network-diagnostics/network-check-target-24w52: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:37:15.836747 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:15.836595    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg podName:8199ec27-52db-463d-acf9-515c99931822 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:47.836577219 +0000 UTC m=+66.240518429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-pgrjg" (UniqueName: "kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg") pod "network-check-target-24w52" (UID: "8199ec27-52db-463d-acf9-515c99931822") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 17:37:16.132189 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:16.132108    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:16.132189 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:16.132132    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:16.132439 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:16.132228    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-24w52" podUID="8199ec27-52db-463d-acf9-515c99931822"
Jul 24 17:37:16.132439 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:16.132369    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:37:17.131335 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.131300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:17.131905 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:17.131420    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-57pv5" podUID="76828591-503d-4d58-87cb-a570db5fc73f"
Jul 24 17:37:17.450656 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.450622    2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-161.ec2.internal" event="NodeReady"
Jul 24 17:37:17.450826 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.450784    2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 17:37:17.517565 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.517523    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-sfqpz"]
Jul 24 17:37:17.522136 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.522097    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tzltv"]
Jul 24 17:37:17.522324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.522194    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.525594 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.525220    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.526197 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.526174    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 17:37:17.526197 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.526189    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 17:37:17.526361 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.526242    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7mn5h\""
Jul 24 17:37:17.530824 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.530543    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 17:37:17.530824 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.530592    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 17:37:17.531050 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.530951    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qpkwg\""
Jul 24 17:37:17.531102 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.531092    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 17:37:17.532240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.532220    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sfqpz"]
Jul 24 17:37:17.535718 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.535693    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzltv"]
Jul 24 17:37:17.651737 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.651706    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07bacd15-46fd-4cdf-b7a1-02ae5ff7086c-cert\") pod \"ingress-canary-tzltv\" (UID: \"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c\") " pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.651932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.651748    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.651932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.651809    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3042e4e-4106-4f3c-be76-9677db69bbf2-config-volume\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.651932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.651842    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64rpq\" (UniqueName: \"kubernetes.io/projected/07bacd15-46fd-4cdf-b7a1-02ae5ff7086c-kube-api-access-64rpq\") pod \"ingress-canary-tzltv\" (UID: \"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c\") " pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.651932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.651902    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3042e4e-4106-4f3c-be76-9677db69bbf2-tmp-dir\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.651932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.651930    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckcxt\" (UniqueName: \"kubernetes.io/projected/f3042e4e-4106-4f3c-be76-9677db69bbf2-kube-api-access-ckcxt\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.753203 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753106    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.753203 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753170    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3042e4e-4106-4f3c-be76-9677db69bbf2-config-volume\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.753203 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64rpq\" (UniqueName: \"kubernetes.io/projected/07bacd15-46fd-4cdf-b7a1-02ae5ff7086c-kube-api-access-64rpq\") pod \"ingress-canary-tzltv\" (UID: \"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c\") " pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.753487 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753237    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3042e4e-4106-4f3c-be76-9677db69bbf2-tmp-dir\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.753487 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753285    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckcxt\" (UniqueName: \"kubernetes.io/projected/f3042e4e-4106-4f3c-be76-9677db69bbf2-kube-api-access-ckcxt\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.753487 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:17.753316    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:17.753487 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:17.753397    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:18.25337369 +0000 UTC m=+36.657314916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:17.753487 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753327    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07bacd15-46fd-4cdf-b7a1-02ae5ff7086c-cert\") pod \"ingress-canary-tzltv\" (UID: \"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c\") " pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.753730 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753691    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3042e4e-4106-4f3c-be76-9677db69bbf2-tmp-dir\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.753914 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.753889    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f3042e4e-4106-4f3c-be76-9677db69bbf2-config-volume\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.758095 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.758068    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07bacd15-46fd-4cdf-b7a1-02ae5ff7086c-cert\") pod \"ingress-canary-tzltv\" (UID: \"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c\") " pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.765296 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.765242    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckcxt\" (UniqueName: \"kubernetes.io/projected/f3042e4e-4106-4f3c-be76-9677db69bbf2-kube-api-access-ckcxt\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:17.765451 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.765365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64rpq\" (UniqueName: \"kubernetes.io/projected/07bacd15-46fd-4cdf-b7a1-02ae5ff7086c-kube-api-access-64rpq\") pod \"ingress-canary-tzltv\" (UID: \"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c\") " pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:17.847396 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:17.847354    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tzltv"
Jul 24 17:37:18.132297 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.132183    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:18.132297 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.132231    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:18.135048 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.134948    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:37:18.135048 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.134979    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:37:18.135048 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.135033    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:37:18.135304 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.135137    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88pnr\""
Jul 24 17:37:18.135304 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.134980    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lvv7h\""
Jul 24 17:37:18.258313 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.258282    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:18.258498 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:18.258456    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:18.258563 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:18.258544    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:19.258525756 +0000 UTC m=+37.662466979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:18.508526 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:18.508308    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tzltv"]
Jul 24 17:37:18.514034 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:37:18.514006    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bacd15_46fd_4cdf_b7a1_02ae5ff7086c.slice/crio-988be5cf35d45c9fa429b6d03b299e47ca7175ab31dd35dc1b60126798b89d07 WatchSource:0}: Error finding container 988be5cf35d45c9fa429b6d03b299e47ca7175ab31dd35dc1b60126798b89d07: Status 404 returned error can't find the container with id 988be5cf35d45c9fa429b6d03b299e47ca7175ab31dd35dc1b60126798b89d07
Jul 24 17:37:19.132048 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:19.132018    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:19.134682 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:19.134645    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:37:19.267361 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:19.267322    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:19.267551 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:19.267447    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:19.267551 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:19.267510    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:21.267493412 +0000 UTC m=+39.671434627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:19.414870 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:19.414778    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzltv" event={"ID":"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c","Type":"ContainerStarted","Data":"988be5cf35d45c9fa429b6d03b299e47ca7175ab31dd35dc1b60126798b89d07"}
Jul 24 17:37:19.418032 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:19.417998    2575 generic.go:356] "Generic (PLEG): container finished" podID="78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58" containerID="8d6005cec18fd8703d45bb51efc4b2b5041980ea6235eab4e15645b842f2d3e6" exitCode=0
Jul 24 17:37:19.418171 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:19.418047    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerDied","Data":"8d6005cec18fd8703d45bb51efc4b2b5041980ea6235eab4e15645b842f2d3e6"}
Jul 24 17:37:20.424630 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:20.424596    2575 generic.go:356] "Generic (PLEG): container finished" podID="78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58" containerID="8de44200aecfaec339b53bc55b14bf26b1208635c2b21a3b90a65177fd329187" exitCode=0
Jul 24 17:37:20.425050 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:20.424645    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerDied","Data":"8de44200aecfaec339b53bc55b14bf26b1208635c2b21a3b90a65177fd329187"}
Jul 24 17:37:21.282379 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:21.282163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:21.282563 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:21.282340    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:21.282563 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:21.282480    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:25.282462954 +0000 UTC m=+43.686404174 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:21.430765 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:21.430729    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" event={"ID":"78ca0b7a-0d9d-4bff-82fb-6bbc80ad0b58","Type":"ContainerStarted","Data":"e5d24bbf9027457602ca95e718744f2386f6e43ffe9ff00f2f349f25ecf4af22"}
Jul 24 17:37:21.432063 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:21.432040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tzltv" event={"ID":"07bacd15-46fd-4cdf-b7a1-02ae5ff7086c","Type":"ContainerStarted","Data":"f1e321eaa7bef3fbbfa92306214900b1b2d252fc10834a498356056e51b5d9f2"}
Jul 24 17:37:21.453038 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:21.452983    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zdmf6" podStartSLOduration=8.586372027 podStartE2EDuration="38.452967691s" totalImagesPullingTime="29.866595664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:21.452166243 +0000 UTC m=+39.856107502" watchObservedRunningTime="2026-07-24 17:37:21.452967691 +0000 UTC m=+39.856908919"
Jul 24 17:37:21.466802 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:21.466754    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tzltv" podStartSLOduration=2.279280356 podStartE2EDuration="4.466735547s" totalImagesPullingTime="2.187455191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:21.466375708 +0000 UTC m=+39.870316938" watchObservedRunningTime="2026-07-24 17:37:21.466735547 +0000 UTC m=+39.870676775"
Jul 24 17:37:25.310672 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:25.310622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:25.311130 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:25.310774    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:25.311130 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:25.310838    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:33.310822165 +0000 UTC m=+51.714763396 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:33.371140 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:33.371095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:33.371652 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:33.371249    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:33.371652 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:33.371334    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:37:49.371315348 +0000 UTC m=+67.775256575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:42.411651 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:42.411621    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-92zjc"
Jul 24 17:37:47.771824 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.771786    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:47.772389 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.771849    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:37:47.774515 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.774491    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 17:37:47.774734 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.774545    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 17:37:47.782514 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:47.782484    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:37:47.782632 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:47.782568    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:38:51.782549105 +0000 UTC m=+130.186490312 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : secret "metrics-daemon-secret" not found
Jul 24 17:37:47.786524 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.786497    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76828591-503d-4d58-87cb-a570db5fc73f-original-pull-secret\") pod \"global-pull-secret-syncer-57pv5\" (UID: \"76828591-503d-4d58-87cb-a570db5fc73f\") " pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:47.872791 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.872750    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:47.875374 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.875352    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 17:37:47.885932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.885900    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 17:37:47.896130 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.896098    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgrjg\" (UniqueName: \"kubernetes.io/projected/8199ec27-52db-463d-acf9-515c99931822-kube-api-access-pgrjg\") pod \"network-check-target-24w52\" (UID: \"8199ec27-52db-463d-acf9-515c99931822\") " pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:47.943332 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:47.943296    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-57pv5"
Jul 24 17:37:48.070002 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:48.069971    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-57pv5"]
Jul 24 17:37:48.073946 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:37:48.073902    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76828591_503d_4d58_87cb_a570db5fc73f.slice/crio-c8f4c095625273c2fc1804a7305dd7a0c907a878962d78cc4e9f7deffd6272a5 WatchSource:0}: Error finding container c8f4c095625273c2fc1804a7305dd7a0c907a878962d78cc4e9f7deffd6272a5: Status 404 returned error can't find the container with id c8f4c095625273c2fc1804a7305dd7a0c907a878962d78cc4e9f7deffd6272a5
Jul 24 17:37:48.154089 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:48.154062    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-88pnr\""
Jul 24 17:37:48.162019 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:48.161983    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:48.284723 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:48.284645    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-24w52"]
Jul 24 17:37:48.287475 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:37:48.287444    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8199ec27_52db_463d_acf9_515c99931822.slice/crio-1b39c420855b1bce2402e0983551aefa57738e6de53d337291e0c16e2abacf2d WatchSource:0}: Error finding container 1b39c420855b1bce2402e0983551aefa57738e6de53d337291e0c16e2abacf2d: Status 404 returned error can't find the container with id 1b39c420855b1bce2402e0983551aefa57738e6de53d337291e0c16e2abacf2d
Jul 24 17:37:48.494078 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:48.494040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-24w52" event={"ID":"8199ec27-52db-463d-acf9-515c99931822","Type":"ContainerStarted","Data":"1b39c420855b1bce2402e0983551aefa57738e6de53d337291e0c16e2abacf2d"}
Jul 24 17:37:48.495374 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:48.495344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-57pv5" event={"ID":"76828591-503d-4d58-87cb-a570db5fc73f","Type":"ContainerStarted","Data":"c8f4c095625273c2fc1804a7305dd7a0c907a878962d78cc4e9f7deffd6272a5"}
Jul 24 17:37:49.383282 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:49.383222    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:37:49.383778 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:49.383380    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:37:49.383778 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:37:49.383452    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:38:21.383429774 +0000 UTC m=+99.787371000 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:37:53.509835 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:53.509791    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-24w52" event={"ID":"8199ec27-52db-463d-acf9-515c99931822","Type":"ContainerStarted","Data":"119e548c3869bc976b89fd963ada73de9c1ec79d82e1ec5c7117e3fbefcc08a9"}
Jul 24 17:37:53.510412 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:53.509878    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:37:53.511173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:53.511151    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-57pv5" event={"ID":"76828591-503d-4d58-87cb-a570db5fc73f","Type":"ContainerStarted","Data":"bf225e1293bbec43779af5e49bb62130e1d8d139df37c84b82e290445b6f99fe"}
Jul 24 17:37:53.524839 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:53.524789    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-24w52" podStartSLOduration=66.194054778 podStartE2EDuration="1m10.524773845s" totalImagesPullingTime="4.330719067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:53.523483901 +0000 UTC m=+71.927425132" watchObservedRunningTime="2026-07-24 17:37:53.524773845 +0000 UTC m=+71.928715073"
Jul 24 17:37:53.539142 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:37:53.539088    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-57pv5" podStartSLOduration=65.990540378 podStartE2EDuration="1m10.539071667s" totalImagesPullingTime="4.548531289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:37:53.538850627 +0000 UTC m=+71.942791857" watchObservedRunningTime="2026-07-24 17:37:53.539071667 +0000 UTC m=+71.943012896"
Jul 24 17:38:21.402511 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:38:21.402471    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:38:21.403018 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:38:21.402599    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:38:21.403018 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:38:21.402661    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:39:25.402647667 +0000 UTC m=+163.806588874 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:38:24.515651 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:38:24.515617    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-24w52"
Jul 24 17:38:51.799278 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:38:51.799233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:38:51.799779 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:38:51.799363    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:38:51.799779 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:38:51.799417    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:53.799401688 +0000 UTC m=+252.203342895 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : secret "metrics-daemon-secret" not found
Jul 24 17:39:20.539679 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:39:20.539629    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-sfqpz" podUID="f3042e4e-4106-4f3c-be76-9677db69bbf2"
Jul 24 17:39:20.715970 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:39:20.715939    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:39:21.146003 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:39:21.145955    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:39:25.437774 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:39:25.437729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:39:25.438191 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:39:25.437840    2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 17:39:25.438191 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:39:25.437925    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls podName:f3042e4e-4106-4f3c-be76-9677db69bbf2 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:27.437905825 +0000 UTC m=+285.841847049 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls") pod "dns-default-sfqpz" (UID: "f3042e4e-4106-4f3c-be76-9677db69bbf2") : secret "dns-default-metrics-tls" not found
Jul 24 17:39:32.133400 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:39:32.133359    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:40:44.813758 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.813720    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-2lg9d"]
Jul 24 17:40:44.815776 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.815754    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"]
Jul 24 17:40:44.815911 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.815874    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:44.817666 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.817645    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-657bdf55cc-7jqkw"]
Jul 24 17:40:44.817792 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.817752    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"
Jul 24 17:40:44.817991 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.817972    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 24 17:40:44.818212 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.818141    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wmjgr\""
Jul 24 17:40:44.818353 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.818226    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:44.818353 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.818226    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:44.818353 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.818285    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 24 17:40:44.819460 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.819444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:44.820129 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.820113    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tzdq8\""
Jul 24 17:40:44.821401 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.821384    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 24 17:40:44.821497 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.821413    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-bmngz\""
Jul 24 17:40:44.821745 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.821712    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 24 17:40:44.821745 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.821721    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 24 17:40:44.822085 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.822067    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 24 17:40:44.822161 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.822071    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 17:40:44.822346 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.822333    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 17:40:44.825338 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.825321    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 24 17:40:44.836415 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.836390    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-2lg9d"]
Jul 24 17:40:44.843975 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.843940    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-657bdf55cc-7jqkw"]
Jul 24 17:40:44.851430 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.851396    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"]
Jul 24 17:40:44.902779 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.902741    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"]
Jul 24 17:40:44.904888 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.904872    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:44.906937 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.906910    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 24 17:40:44.907068 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.906965    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 24 17:40:44.907211 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.907197    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-lbj2z\""
Jul 24 17:40:44.907313 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.907293    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:44.907421 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.907408    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 24 17:40:44.910106 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.910084    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-rcfqt"]
Jul 24 17:40:44.911895 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.911878    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:44.914173 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.914158    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 24 17:40:44.914245 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.914200    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 17:40:44.914377 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.914360    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 17:40:44.914608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.914594    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-znp6s\""
Jul 24 17:40:44.914684 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.914670    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 24 17:40:44.919488 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.919467    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 24 17:40:44.920414 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.920392    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"]
Jul 24 17:40:44.925723 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.925698    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-rcfqt"]
Jul 24 17:40:44.968932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.968894    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-stats-auth\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:44.968932 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.968932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-default-certificate\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:44.969166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.968952    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5de17fc-1e9b-44c7-9096-8b419c33c108-serving-cert\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:44.969166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969016    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6wh5\" (UniqueName: \"kubernetes.io/projected/663e6631-6566-4b7b-9518-da24c85fc15b-kube-api-access-q6wh5\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:44.969166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969061    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/663e6631-6566-4b7b-9518-da24c85fc15b-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:44.969166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969082    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5de17fc-1e9b-44c7-9096-8b419c33c108-config\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:44.969166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969115    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:44.969380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969184    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5de17fc-1e9b-44c7-9096-8b419c33c108-trusted-ca\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:44.969380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969205    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg7vq\" (UniqueName: \"kubernetes.io/projected/f5de17fc-1e9b-44c7-9096-8b419c33c108-kube-api-access-jg7vq\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:44.969380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969222    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bhsr\" (UniqueName: \"kubernetes.io/projected/fba53f5c-2b5b-4b91-abed-426505542cc9-kube-api-access-4bhsr\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:44.969380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969243    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:44.969380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969261    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5khcb\" (UniqueName: \"kubernetes.io/projected/feff2bfb-e972-43ef-bb70-0b94eb48e918-kube-api-access-5khcb\") pod \"network-check-source-7d88cb55d8-djj62\" (UID: \"feff2bfb-e972-43ef-bb70-0b94eb48e918\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"
Jul 24 17:40:44.969380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:44.969307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/663e6631-6566-4b7b-9518-da24c85fc15b-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.070292 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070171    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5de17fc-1e9b-44c7-9096-8b419c33c108-trusted-ca\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.070292 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070216    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg7vq\" (UniqueName: \"kubernetes.io/projected/f5de17fc-1e9b-44c7-9096-8b419c33c108-kube-api-access-jg7vq\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.070292 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070240    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bhsr\" (UniqueName: \"kubernetes.io/projected/fba53f5c-2b5b-4b91-abed-426505542cc9-kube-api-access-4bhsr\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.070292 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070283    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/970c6912-05fb-4fd7-a428-407244b542fc-snapshots\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.070591 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/970c6912-05fb-4fd7-a428-407244b542fc-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.070591 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070541    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.070591 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070570    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5khcb\" (UniqueName: \"kubernetes.io/projected/feff2bfb-e972-43ef-bb70-0b94eb48e918-kube-api-access-5khcb\") pod \"network-check-source-7d88cb55d8-djj62\" (UID: \"feff2bfb-e972-43ef-bb70-0b94eb48e918\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"
Jul 24 17:40:45.070751 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070596    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/663e6631-6566-4b7b-9518-da24c85fc15b-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.070751 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070623    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrk54\" (UniqueName: \"kubernetes.io/projected/970c6912-05fb-4fd7-a428-407244b542fc-kube-api-access-hrk54\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.070751 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/970c6912-05fb-4fd7-a428-407244b542fc-service-ca-bundle\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.070751 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:45.070675    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:45.070751 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-stats-auth\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.070959 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:45.070763    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:45.570743909 +0000 UTC m=+243.974685121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : secret "router-metrics-certs-default" not found
Jul 24 17:40:45.070959 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070795    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/970c6912-05fb-4fd7-a428-407244b542fc-tmp\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.070959 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-default-certificate\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.070959 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070873    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5de17fc-1e9b-44c7-9096-8b419c33c108-serving-cert\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.070959 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070903    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6wh5\" (UniqueName: \"kubernetes.io/projected/663e6631-6566-4b7b-9518-da24c85fc15b-kube-api-access-q6wh5\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.070959 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/970c6912-05fb-4fd7-a428-407244b542fc-serving-cert\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.071287 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.070992    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/663e6631-6566-4b7b-9518-da24c85fc15b-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.071287 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.071026    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5de17fc-1e9b-44c7-9096-8b419c33c108-config\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.071287 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.071057    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.071287 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:45.071179    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:45.571163838 +0000 UTC m=+243.975105045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:45.071287 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.071203    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f5de17fc-1e9b-44c7-9096-8b419c33c108-trusted-ca\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.071609 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.071587    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/663e6631-6566-4b7b-9518-da24c85fc15b-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.072047 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.072022    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5de17fc-1e9b-44c7-9096-8b419c33c108-config\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.073196 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.073170    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/663e6631-6566-4b7b-9518-da24c85fc15b-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.073301 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.073179    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5de17fc-1e9b-44c7-9096-8b419c33c108-serving-cert\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.073301 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.073253    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-default-certificate\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.073419 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.073402    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-stats-auth\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.078660 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.078635    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6wh5\" (UniqueName: \"kubernetes.io/projected/663e6631-6566-4b7b-9518-da24c85fc15b-kube-api-access-q6wh5\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k\" (UID: \"663e6631-6566-4b7b-9518-da24c85fc15b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.079144 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.079126    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bhsr\" (UniqueName: \"kubernetes.io/projected/fba53f5c-2b5b-4b91-abed-426505542cc9-kube-api-access-4bhsr\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.079833 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.079809    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5khcb\" (UniqueName: \"kubernetes.io/projected/feff2bfb-e972-43ef-bb70-0b94eb48e918-kube-api-access-5khcb\") pod \"network-check-source-7d88cb55d8-djj62\" (UID: \"feff2bfb-e972-43ef-bb70-0b94eb48e918\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"
Jul 24 17:40:45.080243 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.080223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg7vq\" (UniqueName: \"kubernetes.io/projected/f5de17fc-1e9b-44c7-9096-8b419c33c108-kube-api-access-jg7vq\") pod \"console-operator-584dcb6b54-2lg9d\" (UID: \"f5de17fc-1e9b-44c7-9096-8b419c33c108\") " pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.127699 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.127656    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:45.131459 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.131439    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"
Jul 24 17:40:45.172298 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.172084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/970c6912-05fb-4fd7-a428-407244b542fc-snapshots\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.172298 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.172120    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/970c6912-05fb-4fd7-a428-407244b542fc-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.172298 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.172150    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrk54\" (UniqueName: \"kubernetes.io/projected/970c6912-05fb-4fd7-a428-407244b542fc-kube-api-access-hrk54\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.172298 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.172175    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/970c6912-05fb-4fd7-a428-407244b542fc-service-ca-bundle\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.172298 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.172196    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/970c6912-05fb-4fd7-a428-407244b542fc-tmp\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.172298 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.172214    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/970c6912-05fb-4fd7-a428-407244b542fc-serving-cert\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.173496 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.173464    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/970c6912-05fb-4fd7-a428-407244b542fc-snapshots\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.173728 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.173693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/970c6912-05fb-4fd7-a428-407244b542fc-tmp\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.173786 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.173768    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/970c6912-05fb-4fd7-a428-407244b542fc-service-ca-bundle\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.174195 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.174010    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/970c6912-05fb-4fd7-a428-407244b542fc-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.175608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.175581    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/970c6912-05fb-4fd7-a428-407244b542fc-serving-cert\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.182863 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.182114    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrk54\" (UniqueName: \"kubernetes.io/projected/970c6912-05fb-4fd7-a428-407244b542fc-kube-api-access-hrk54\") pod \"insights-operator-5885fdb759-rcfqt\" (UID: \"970c6912-05fb-4fd7-a428-407244b542fc\") " pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.214181 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.213954    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"
Jul 24 17:40:45.221971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.221937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-rcfqt"
Jul 24 17:40:45.269956 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.269921    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62"]
Jul 24 17:40:45.275826 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:40:45.275796    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeff2bfb_e972_43ef_bb70_0b94eb48e918.slice/crio-825c56923140a1b6688e688da484e28e15da97a20a359128c926233840c3cbf0 WatchSource:0}: Error finding container 825c56923140a1b6688e688da484e28e15da97a20a359128c926233840c3cbf0: Status 404 returned error can't find the container with id 825c56923140a1b6688e688da484e28e15da97a20a359128c926233840c3cbf0
Jul 24 17:40:45.296864 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.296767    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-2lg9d"]
Jul 24 17:40:45.300877 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:40:45.300848    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5de17fc_1e9b_44c7_9096_8b419c33c108.slice/crio-e53898d39efd460393e24129243b14ef78c3b219036cf2418ab27e0bf0ff0602 WatchSource:0}: Error finding container e53898d39efd460393e24129243b14ef78c3b219036cf2418ab27e0bf0ff0602: Status 404 returned error can't find the container with id e53898d39efd460393e24129243b14ef78c3b219036cf2418ab27e0bf0ff0602
Jul 24 17:40:45.367369 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.367149    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k"]
Jul 24 17:40:45.369785 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:40:45.369756    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663e6631_6566_4b7b_9518_da24c85fc15b.slice/crio-52d82bcc2e31ca9a8f6429be1329bedffeb33b464de0be6269f7cd9870187b4e WatchSource:0}: Error finding container 52d82bcc2e31ca9a8f6429be1329bedffeb33b464de0be6269f7cd9870187b4e: Status 404 returned error can't find the container with id 52d82bcc2e31ca9a8f6429be1329bedffeb33b464de0be6269f7cd9870187b4e
Jul 24 17:40:45.381644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.381612    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-rcfqt"]
Jul 24 17:40:45.384612 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:40:45.384582    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod970c6912_05fb_4fd7_a428_407244b542fc.slice/crio-a058722bcd35719d237f86f52073e0781f37fef8fd0195386c7bf19845e9a184 WatchSource:0}: Error finding container a058722bcd35719d237f86f52073e0781f37fef8fd0195386c7bf19845e9a184: Status 404 returned error can't find the container with id a058722bcd35719d237f86f52073e0781f37fef8fd0195386c7bf19845e9a184
Jul 24 17:40:45.576174 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.576072    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.576174 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.576143    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:45.576395 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:45.576223    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:45.576395 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:45.576258    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:46.576243971 +0000 UTC m=+244.980185177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:45.576395 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:45.576296    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:46.576288114 +0000 UTC m=+244.980229376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : secret "router-metrics-certs-default" not found
Jul 24 17:40:45.924318 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.924130    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62" event={"ID":"feff2bfb-e972-43ef-bb70-0b94eb48e918","Type":"ContainerStarted","Data":"1cb3808a137c88fa3cd65bec61c2390241290223263e8e623b106239cee3b1d3"}
Jul 24 17:40:45.924318 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.924196    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62" event={"ID":"feff2bfb-e972-43ef-bb70-0b94eb48e918","Type":"ContainerStarted","Data":"825c56923140a1b6688e688da484e28e15da97a20a359128c926233840c3cbf0"}
Jul 24 17:40:45.925901 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.925869    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" event={"ID":"f5de17fc-1e9b-44c7-9096-8b419c33c108","Type":"ContainerStarted","Data":"e53898d39efd460393e24129243b14ef78c3b219036cf2418ab27e0bf0ff0602"}
Jul 24 17:40:45.927335 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.927299    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-rcfqt" event={"ID":"970c6912-05fb-4fd7-a428-407244b542fc","Type":"ContainerStarted","Data":"a058722bcd35719d237f86f52073e0781f37fef8fd0195386c7bf19845e9a184"}
Jul 24 17:40:45.928857 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.928806    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k" event={"ID":"663e6631-6566-4b7b-9518-da24c85fc15b","Type":"ContainerStarted","Data":"52d82bcc2e31ca9a8f6429be1329bedffeb33b464de0be6269f7cd9870187b4e"}
Jul 24 17:40:45.941209 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:45.940921    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-djj62" podStartSLOduration=1.9408970939999999 podStartE2EDuration="1.940897094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:45.939884807 +0000 UTC m=+244.343826037" watchObservedRunningTime="2026-07-24 17:40:45.940897094 +0000 UTC m=+244.344838324"
Jul 24 17:40:46.585996 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:46.585645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:46.585996 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:46.585748    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:46.585996 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:46.585845    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:46.585996 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:46.585891    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:48.585872412 +0000 UTC m=+246.989813622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:46.585996 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:46.585912    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:48.585902654 +0000 UTC m=+246.989843863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : secret "router-metrics-certs-default" not found
Jul 24 17:40:48.603039 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.602934    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:48.603039 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.602998    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:48.603500 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:48.603090    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:48.603500 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:48.603140    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:52.603115482 +0000 UTC m=+251.007056695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:48.603500 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:48.603171    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:52.60316197 +0000 UTC m=+251.007103185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : secret "router-metrics-certs-default" not found
Jul 24 17:40:48.847168 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.847128    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l4n5p"]
Jul 24 17:40:48.849203 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.849182    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:48.851538 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.851515    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cpj25\""
Jul 24 17:40:48.851538 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.851524    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 17:40:48.851538 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.851515    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 17:40:48.863170 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.863141    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l4n5p"]
Jul 24 17:40:48.940766 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.940737    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/0.log"
Jul 24 17:40:48.940949 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.940781    2575 generic.go:356] "Generic (PLEG): container finished" podID="f5de17fc-1e9b-44c7-9096-8b419c33c108" containerID="94c38c23306377323a4972632ab8c0f68e913d93edcc27d3002bcd8d52f85523" exitCode=255
Jul 24 17:40:48.940949 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.940873    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" event={"ID":"f5de17fc-1e9b-44c7-9096-8b419c33c108","Type":"ContainerDied","Data":"94c38c23306377323a4972632ab8c0f68e913d93edcc27d3002bcd8d52f85523"}
Jul 24 17:40:48.941086 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.941073    2575 scope.go:117] "RemoveContainer" containerID="94c38c23306377323a4972632ab8c0f68e913d93edcc27d3002bcd8d52f85523"
Jul 24 17:40:48.942554 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.942531    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-rcfqt" event={"ID":"970c6912-05fb-4fd7-a428-407244b542fc","Type":"ContainerStarted","Data":"6945ce15875012908f61c874cc8b15c4f0b5aad8b120642b800f920543d0c510"}
Jul 24 17:40:48.944249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.944228    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k" event={"ID":"663e6631-6566-4b7b-9518-da24c85fc15b","Type":"ContainerStarted","Data":"dd5a11bab32bfa5fb42fea50d3123fac0c11aba6cc6d5a3054c06af24a711365"}
Jul 24 17:40:48.972625 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.972571    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k" podStartSLOduration=2.100921319 podStartE2EDuration="4.972554709s" totalImagesPullingTime="2.87163339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:48.971832533 +0000 UTC m=+247.375773764" watchObservedRunningTime="2026-07-24 17:40:48.972554709 +0000 UTC m=+247.376495938"
Jul 24 17:40:48.987986 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:48.987933    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-rcfqt" podStartSLOduration=2.132799148 podStartE2EDuration="4.987913746s" totalImagesPullingTime="2.855114598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:48.987167917 +0000 UTC m=+247.391109171" watchObservedRunningTime="2026-07-24 17:40:48.987913746 +0000 UTC m=+247.391854975"
Jul 24 17:40:49.006015 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.005974    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55wmf\" (UniqueName: \"kubernetes.io/projected/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-api-access-55wmf\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.006207 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.006139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.006207 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.006185    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e81d826-1445-4170-b886-0f36ac0b5d00-crio-socket\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.006342 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.006214    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.006342 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.006250    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81d826-1445-4170-b886-0f36ac0b5d00-data-volume\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107346 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107229    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107346 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107300    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e81d826-1445-4170-b886-0f36ac0b5d00-crio-socket\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107346 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107329    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107628 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.107366    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:49.107628 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107389    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7e81d826-1445-4170-b886-0f36ac0b5d00-crio-socket\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107628 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107378    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81d826-1445-4170-b886-0f36ac0b5d00-data-volume\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107628 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.107440    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:49.60741793 +0000 UTC m=+248.011359153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:49.107628 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.107541    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:49.107628 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.107585    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:49.607572523 +0000 UTC m=+248.011513736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:49.107940 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107680    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e81d826-1445-4170-b886-0f36ac0b5d00-data-volume\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.107940 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.107773    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55wmf\" (UniqueName: \"kubernetes.io/projected/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-api-access-55wmf\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.115513 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.115488    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55wmf\" (UniqueName: \"kubernetes.io/projected/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-api-access-55wmf\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.611802 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.611764    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.612206 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.611918    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:49.612206 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.611972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:49.612206 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.611981    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:50.611965141 +0000 UTC m=+249.015906348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:49.612206 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.612055    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:49.612206 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.612097    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:50.612085795 +0000 UTC m=+249.016027002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:49.773653 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.773621    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"]
Jul 24 17:40:49.776435 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.776409    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"
Jul 24 17:40:49.778806 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.778779    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 17:40:49.779559 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.779539    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 17:40:49.779559 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.779550    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vlt7t\""
Jul 24 17:40:49.784624 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.784595    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"]
Jul 24 17:40:49.914897 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.914801    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqdf2\" (UniqueName: \"kubernetes.io/projected/edc96739-ff51-4de0-8014-fe3165be4989-kube-api-access-vqdf2\") pod \"migrator-6865c9967-m4w77\" (UID: \"edc96739-ff51-4de0-8014-fe3165be4989\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"
Jul 24 17:40:49.948935 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.948907    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:40:49.949337 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.949319    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/0.log"
Jul 24 17:40:49.949423 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.949362    2575 generic.go:356] "Generic (PLEG): container finished" podID="f5de17fc-1e9b-44c7-9096-8b419c33c108" containerID="5a2850d9c6ac1436f466add9a45b086f997310d94a488ffd6f7382374aec0e77" exitCode=255
Jul 24 17:40:49.949486 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.949441    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" event={"ID":"f5de17fc-1e9b-44c7-9096-8b419c33c108","Type":"ContainerDied","Data":"5a2850d9c6ac1436f466add9a45b086f997310d94a488ffd6f7382374aec0e77"}
Jul 24 17:40:49.949543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.949489    2575 scope.go:117] "RemoveContainer" containerID="94c38c23306377323a4972632ab8c0f68e913d93edcc27d3002bcd8d52f85523"
Jul 24 17:40:49.949728 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:49.949705    2575 scope.go:117] "RemoveContainer" containerID="5a2850d9c6ac1436f466add9a45b086f997310d94a488ffd6f7382374aec0e77"
Jul 24 17:40:49.949945 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:49.949921    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-2lg9d_openshift-console-operator(f5de17fc-1e9b-44c7-9096-8b419c33c108)\"" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" podUID="f5de17fc-1e9b-44c7-9096-8b419c33c108"
Jul 24 17:40:50.015435 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.015397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqdf2\" (UniqueName: \"kubernetes.io/projected/edc96739-ff51-4de0-8014-fe3165be4989-kube-api-access-vqdf2\") pod \"migrator-6865c9967-m4w77\" (UID: \"edc96739-ff51-4de0-8014-fe3165be4989\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"
Jul 24 17:40:50.023589 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.023546    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqdf2\" (UniqueName: \"kubernetes.io/projected/edc96739-ff51-4de0-8014-fe3165be4989-kube-api-access-vqdf2\") pod \"migrator-6865c9967-m4w77\" (UID: \"edc96739-ff51-4de0-8014-fe3165be4989\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"
Jul 24 17:40:50.086650 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.086616    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"
Jul 24 17:40:50.219453 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.219365    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77"]
Jul 24 17:40:50.222098 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:40:50.222063    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedc96739_ff51_4de0_8014_fe3165be4989.slice/crio-1410c7b2144573c9ee730a1579d7f0f0c0e91f4ee630126fc7669b6ae6a6cfc8 WatchSource:0}: Error finding container 1410c7b2144573c9ee730a1579d7f0f0c0e91f4ee630126fc7669b6ae6a6cfc8: Status 404 returned error can't find the container with id 1410c7b2144573c9ee730a1579d7f0f0c0e91f4ee630126fc7669b6ae6a6cfc8
Jul 24 17:40:50.620946 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.620911    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:50.620946 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.620953    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:50.621544 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:50.621062    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:50.621544 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:50.621134    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:50.621544 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:50.621153    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:52.621130647 +0000 UTC m=+251.025071855 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:50.621544 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:50.621186    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:52.621172424 +0000 UTC m=+251.025113632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:50.631319 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.631289    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-59zmp_3855ca0f-6f2c-4fbc-b018-b5b0033ae11d/dns-node-resolver/0.log"
Jul 24 17:40:50.955811 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.955775    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77" event={"ID":"edc96739-ff51-4de0-8014-fe3165be4989","Type":"ContainerStarted","Data":"1410c7b2144573c9ee730a1579d7f0f0c0e91f4ee630126fc7669b6ae6a6cfc8"}
Jul 24 17:40:50.957359 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.957337    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:40:50.957739 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:50.957717    2575 scope.go:117] "RemoveContainer" containerID="5a2850d9c6ac1436f466add9a45b086f997310d94a488ffd6f7382374aec0e77"
Jul 24 17:40:50.957925 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:50.957900    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-2lg9d_openshift-console-operator(f5de17fc-1e9b-44c7-9096-8b419c33c108)\"" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" podUID="f5de17fc-1e9b-44c7-9096-8b419c33c108"
Jul 24 17:40:51.961963 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:51.961925    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77" event={"ID":"edc96739-ff51-4de0-8014-fe3165be4989","Type":"ContainerStarted","Data":"454fd6488e932173cc06323ae2a83e09c9ed710068a241065438c21c86814952"}
Jul 24 17:40:51.962463 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:51.961989    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77" event={"ID":"edc96739-ff51-4de0-8014-fe3165be4989","Type":"ContainerStarted","Data":"aca50f34b2eb94d585ad4c010c15c9c40f8957151a7e874069a228ddfb5ce6b8"}
Jul 24 17:40:51.989284 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:51.989209    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-m4w77" podStartSLOduration=1.934594361 podStartE2EDuration="2.989194046s" totalImagesPullingTime="1.054599685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:40:51.987760543 +0000 UTC m=+250.391701772" watchObservedRunningTime="2026-07-24 17:40:51.989194046 +0000 UTC m=+250.393135274"
Jul 24 17:40:52.032807 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:52.032777    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gtbnj_fe83c62b-750b-4ae3-a4ae-5ca6a056c125/node-ca/0.log"
Jul 24 17:40:52.639016 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:52.638972    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:52.639219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:52.639090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:52.639219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:52.639122    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:40:52.639219 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639132    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:52.639219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:52.639147    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:52.639219 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639198    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:56.639179932 +0000 UTC m=+255.043121140 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:52.639485 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639231    2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 24 17:40:52.639485 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639260    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:52.639485 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639285    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:00.63925093 +0000 UTC m=+259.043192137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : configmap references non-existent config key: service-ca.crt
Jul 24 17:40:52.639485 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639340    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs podName:fba53f5c-2b5b-4b91-abed-426505542cc9 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:00.639326834 +0000 UTC m=+259.043268044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs") pod "router-default-657bdf55cc-7jqkw" (UID: "fba53f5c-2b5b-4b91-abed-426505542cc9") : secret "router-metrics-certs-default" not found
Jul 24 17:40:52.639485 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:52.639355    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:40:56.639346205 +0000 UTC m=+255.043287414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:52.831109 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:52.831079    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzltv_07bacd15-46fd-4cdf-b7a1-02ae5ff7086c/serve-healthcheck-canary/0.log"
Jul 24 17:40:53.849360 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:53.849324    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:40:53.849768 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:53.849492    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 24 17:40:53.849768 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:53.849561    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs podName:cbeb29f5-efec-44dc-aed1-21dc7e02dca4 nodeName:}" failed. No retries permitted until 2026-07-24 17:42:55.849545204 +0000 UTC m=+374.253486411 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs") pod "network-metrics-daemon-x2cmc" (UID: "cbeb29f5-efec-44dc-aed1-21dc7e02dca4") : secret "metrics-daemon-secret" not found
Jul 24 17:40:55.128477 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:55.128423    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:55.128477 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:55.128494    2575 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:40:55.128922 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:55.128852    2575 scope.go:117] "RemoveContainer" containerID="5a2850d9c6ac1436f466add9a45b086f997310d94a488ffd6f7382374aec0e77"
Jul 24 17:40:55.129023 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:55.129001    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-2lg9d_openshift-console-operator(f5de17fc-1e9b-44c7-9096-8b419c33c108)\"" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" podUID="f5de17fc-1e9b-44c7-9096-8b419c33c108"
Jul 24 17:40:56.672090 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:56.672045    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:56.672090 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:40:56.672094    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:40:56.672564 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:56.672166    2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 17:40:56.672564 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:56.672228    2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 17:40:56.672564 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:56.672237    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:04.672219504 +0000 UTC m=+263.076160711 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : configmap "kube-rbac-proxy" not found
Jul 24 17:40:56.672564 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:40:56.672290    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls podName:7e81d826-1445-4170-b886-0f36ac0b5d00 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:04.672260999 +0000 UTC m=+263.076202206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4n5p" (UID: "7e81d826-1445-4170-b886-0f36ac0b5d00") : secret "insights-runtime-extractor-tls" not found
Jul 24 17:41:00.706673 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.706631    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:00.706673 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.706675    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:00.707451 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.707432    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fba53f5c-2b5b-4b91-abed-426505542cc9-service-ca-bundle\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:00.709077 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.709057    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fba53f5c-2b5b-4b91-abed-426505542cc9-metrics-certs\") pod \"router-default-657bdf55cc-7jqkw\" (UID: \"fba53f5c-2b5b-4b91-abed-426505542cc9\") " pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:00.737893 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.737841    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:00.876198 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.876173    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-657bdf55cc-7jqkw"]
Jul 24 17:41:00.878578 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:00.878551    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba53f5c_2b5b_4b91_abed_426505542cc9.slice/crio-9380148b0a09dabaf9ccd45fb4a5d1570cafe5a32dd2457e3b33a179b60b3952 WatchSource:0}: Error finding container 9380148b0a09dabaf9ccd45fb4a5d1570cafe5a32dd2457e3b33a179b60b3952: Status 404 returned error can't find the container with id 9380148b0a09dabaf9ccd45fb4a5d1570cafe5a32dd2457e3b33a179b60b3952
Jul 24 17:41:00.992171 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.992081    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-657bdf55cc-7jqkw" event={"ID":"fba53f5c-2b5b-4b91-abed-426505542cc9","Type":"ContainerStarted","Data":"4c56747d5da7cc6aa153ebcc64916188ac064efb6924cf6ff0ec16aa223462ae"}
Jul 24 17:41:00.992171 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:00.992133    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-657bdf55cc-7jqkw" event={"ID":"fba53f5c-2b5b-4b91-abed-426505542cc9","Type":"ContainerStarted","Data":"9380148b0a09dabaf9ccd45fb4a5d1570cafe5a32dd2457e3b33a179b60b3952"}
Jul 24 17:41:01.018437 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:01.018361    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-657bdf55cc-7jqkw" podStartSLOduration=17.018339838 podStartE2EDuration="17.018339838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:01.017145861 +0000 UTC m=+259.421087085" watchObservedRunningTime="2026-07-24 17:41:01.018339838 +0000 UTC m=+259.422281068"
Jul 24 17:41:01.738252 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:01.738213    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:01.740735 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:01.740713    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:01.995901 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:01.995814    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:01.997096 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:01.997076    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-657bdf55cc-7jqkw"
Jul 24 17:41:04.739280 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:04.739172    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:41:04.739280 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:04.739213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:41:04.739723 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:04.739703    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7e81d826-1445-4170-b886-0f36ac0b5d00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:41:04.741519 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:04.741498    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7e81d826-1445-4170-b886-0f36ac0b5d00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4n5p\" (UID: \"7e81d826-1445-4170-b886-0f36ac0b5d00\") " pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:41:04.759322 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:04.759283    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l4n5p"
Jul 24 17:41:04.889558 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:04.889526    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l4n5p"]
Jul 24 17:41:04.892682 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:04.892647    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e81d826_1445_4170_b886_0f36ac0b5d00.slice/crio-46c2e7c219e39a509c68fe2677ca0edc58da2201b35d5c22ffc03c5da01012bb WatchSource:0}: Error finding container 46c2e7c219e39a509c68fe2677ca0edc58da2201b35d5c22ffc03c5da01012bb: Status 404 returned error can't find the container with id 46c2e7c219e39a509c68fe2677ca0edc58da2201b35d5c22ffc03c5da01012bb
Jul 24 17:41:05.006895 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:05.006813    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4n5p" event={"ID":"7e81d826-1445-4170-b886-0f36ac0b5d00","Type":"ContainerStarted","Data":"46c2e7c219e39a509c68fe2677ca0edc58da2201b35d5c22ffc03c5da01012bb"}
Jul 24 17:41:07.132653 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:07.132618    2575 scope.go:117] "RemoveContainer" containerID="5a2850d9c6ac1436f466add9a45b086f997310d94a488ffd6f7382374aec0e77"
Jul 24 17:41:10.027981 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:10.027904    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:41:10.028423 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:10.028002    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" event={"ID":"f5de17fc-1e9b-44c7-9096-8b419c33c108","Type":"ContainerStarted","Data":"3bed3e37896efcbc4fc7941c889907221d3d76fff70ffdcacc1c59151ec4f3e0"}
Jul 24 17:41:10.028423 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:10.028340    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:41:10.029761 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:10.029739    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4n5p" event={"ID":"7e81d826-1445-4170-b886-0f36ac0b5d00","Type":"ContainerStarted","Data":"a4301476f12865196227107af88388ba9497c79d11d32041a989d1747b284a3b"}
Jul 24 17:41:10.045441 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:10.045389    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d" podStartSLOduration=23.111765517 podStartE2EDuration="26.045373899s" totalImagesPullingTime="2.933608382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:40:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:10.043509459 +0000 UTC m=+268.447450691" watchObservedRunningTime="2026-07-24 17:41:10.045373899 +0000 UTC m=+268.449315128"
Jul 24 17:41:10.757085 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:10.756863    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-2lg9d"
Jul 24 17:41:14.046490 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:14.046444    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4n5p" event={"ID":"7e81d826-1445-4170-b886-0f36ac0b5d00","Type":"ContainerStarted","Data":"80a7f39cbd7a00f2c231216212dc9aae2a56d84133fad1e00140a06acf1af794"}
Jul 24 17:41:16.440926 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.440891    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"]
Jul 24 17:41:16.463023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.462987    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"]
Jul 24 17:41:16.463210 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.463110    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.465963 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.465931    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 24 17:41:16.466456 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.466318    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 17:41:16.466456 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.466413    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qqq9q\""
Jul 24 17:41:16.467164 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.467061    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 17:41:16.467368 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.467321    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 17:41:16.469606 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.469580    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 24 17:41:16.538624 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.538579    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a46d6ad4-81ac-4892-8211-ca55bc65d596-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.538828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.538727    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a46d6ad4-81ac-4892-8211-ca55bc65d596-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.538828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.538783    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv4d4\" (UniqueName: \"kubernetes.io/projected/a46d6ad4-81ac-4892-8211-ca55bc65d596-kube-api-access-zv4d4\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.538951 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.538830    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a46d6ad4-81ac-4892-8211-ca55bc65d596-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.640019 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.639981    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a46d6ad4-81ac-4892-8211-ca55bc65d596-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.640237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.640040    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a46d6ad4-81ac-4892-8211-ca55bc65d596-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.640237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.640090    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a46d6ad4-81ac-4892-8211-ca55bc65d596-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.640237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.640124    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv4d4\" (UniqueName: \"kubernetes.io/projected/a46d6ad4-81ac-4892-8211-ca55bc65d596-kube-api-access-zv4d4\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.640831 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.640802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a46d6ad4-81ac-4892-8211-ca55bc65d596-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.642888 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.642863    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a46d6ad4-81ac-4892-8211-ca55bc65d596-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.643004 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.642914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a46d6ad4-81ac-4892-8211-ca55bc65d596-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.648141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.648080    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv4d4\" (UniqueName: \"kubernetes.io/projected/a46d6ad4-81ac-4892-8211-ca55bc65d596-kube-api-access-zv4d4\") pod \"prometheus-operator-84d69f794d-9n9kc\" (UID: \"a46d6ad4-81ac-4892-8211-ca55bc65d596\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.775229 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.775149    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"
Jul 24 17:41:16.964872 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:16.964826    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-9n9kc"]
Jul 24 17:41:16.967341 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:16.967311    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda46d6ad4_81ac_4892_8211_ca55bc65d596.slice/crio-8ac541f5b260fa32aab1003fdb5294631cbde9e8e06fe8561147ca37bb4afe82 WatchSource:0}: Error finding container 8ac541f5b260fa32aab1003fdb5294631cbde9e8e06fe8561147ca37bb4afe82: Status 404 returned error can't find the container with id 8ac541f5b260fa32aab1003fdb5294631cbde9e8e06fe8561147ca37bb4afe82
Jul 24 17:41:17.057612 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:17.057514    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc" event={"ID":"a46d6ad4-81ac-4892-8211-ca55bc65d596","Type":"ContainerStarted","Data":"8ac541f5b260fa32aab1003fdb5294631cbde9e8e06fe8561147ca37bb4afe82"}
Jul 24 17:41:17.059420 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:17.059394    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4n5p" event={"ID":"7e81d826-1445-4170-b886-0f36ac0b5d00","Type":"ContainerStarted","Data":"3fa4511ed8f6fa8134cb878bac21ab009feebfb4ea4eb5341205557277ab0b29"}
Jul 24 17:41:17.077890 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:17.077841    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-l4n5p" podStartSLOduration=17.196683412 podStartE2EDuration="29.077819145s" totalImagesPullingTime="11.881135733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 17:40:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:17.077519465 +0000 UTC m=+275.481460694" watchObservedRunningTime="2026-07-24 17:41:17.077819145 +0000 UTC m=+275.481760352"
Jul 24 17:41:19.069971 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:19.069934    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc" event={"ID":"a46d6ad4-81ac-4892-8211-ca55bc65d596","Type":"ContainerStarted","Data":"01c01174b523cbc2781ad7d06db920b9426d0e9232595f4c3827df9b1a57c8a4"}
Jul 24 17:41:19.070414 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:19.069984    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc" event={"ID":"a46d6ad4-81ac-4892-8211-ca55bc65d596","Type":"ContainerStarted","Data":"4506cdd5ec9228d413b57e04942db2203e732555669b126e87268cb627bd6a40"}
Jul 24 17:41:19.090721 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:19.090644    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-9n9kc" podStartSLOduration=1.7237632459999999 podStartE2EDuration="3.090626758s" totalImagesPullingTime="1.366863512s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:19.089577622 +0000 UTC m=+277.493518852" watchObservedRunningTime="2026-07-24 17:41:19.090626758 +0000 UTC m=+277.494567987"
Jul 24 17:41:20.856427 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.856387    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"]
Jul 24 17:41:20.859876 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.859851    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.860028 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.859998    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v6jgr"]
Jul 24 17:41:20.861982 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.861962    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 17:41:20.862108 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.861967    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9js5b\""
Jul 24 17:41:20.862698 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.862679    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 17:41:20.863809 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.863649    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 17:41:20.863809 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.863765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.866534 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.866504    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 17:41:20.866669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.866655    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xfx6g\""
Jul 24 17:41:20.866726 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.866712    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 17:41:20.866912 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.866898    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 17:41:20.874259 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.874233    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"]
Jul 24 17:41:20.972199 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972167    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.972382 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972232    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6e1d3688-4227-4f08-90ab-642c12dd03fd-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.972382 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6l6b\" (UniqueName: \"kubernetes.io/projected/71cbfa0f-d3dc-43c9-9488-8a3953536703-kube-api-access-d6l6b\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972382 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972349    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.972382 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-wtmp\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.972543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972424    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d3688-4227-4f08-90ab-642c12dd03fd-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.972543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972448    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kznh6\" (UniqueName: \"kubernetes.io/projected/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-api-access-kznh6\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:20.972543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972478    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-root\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972509    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972716 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972542    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-sys\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972716 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972577    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-textfile\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972716 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972622    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71cbfa0f-d3dc-43c9-9488-8a3953536703-metrics-client-ca\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972716 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972642    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:20.972716 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:20.972658    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-accelerators-collector-config\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.073584 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073597    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d3688-4227-4f08-90ab-642c12dd03fd-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073622    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kznh6\" (UniqueName: \"kubernetes.io/projected/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-api-access-kznh6\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073649    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-root\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073719    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-sys\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-textfile\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.073790 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71cbfa0f-d3dc-43c9-9488-8a3953536703-metrics-client-ca\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073827    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-accelerators-collector-config\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073865    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6e1d3688-4227-4f08-90ab-642c12dd03fd-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.073961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6l6b\" (UniqueName: \"kubernetes.io/projected/71cbfa0f-d3dc-43c9-9488-8a3953536703-kube-api-access-d6l6b\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.074004    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.074141 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.074034    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-wtmp\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074533 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.074191    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-wtmp\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074533 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.074444    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-root\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074630 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:21.074535    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 17:41:21.074630 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:21.074589    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls podName:71cbfa0f-d3dc-43c9-9488-8a3953536703 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:21.574569504 +0000 UTC m=+279.978510725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls") pod "node-exporter-v6jgr" (UID: "71cbfa0f-d3dc-43c9-9488-8a3953536703") : secret "node-exporter-tls" not found
Jul 24 17:41:21.074837 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.074813    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71cbfa0f-d3dc-43c9-9488-8a3953536703-sys\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.074941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.074916    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6e1d3688-4227-4f08-90ab-642c12dd03fd-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.075108 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.075071    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-textfile\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.075401 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.075368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/6e1d3688-4227-4f08-90ab-642c12dd03fd-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.076158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.076138    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.083907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.076338    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/71cbfa0f-d3dc-43c9-9488-8a3953536703-metrics-client-ca\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.083907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.076376    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-accelerators-collector-config\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.083907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.078671    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.083907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.083618    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.083907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.083770    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.086763 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.086737    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6l6b\" (UniqueName: \"kubernetes.io/projected/71cbfa0f-d3dc-43c9-9488-8a3953536703-kube-api-access-d6l6b\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.086910 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.086855    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kznh6\" (UniqueName: \"kubernetes.io/projected/6e1d3688-4227-4f08-90ab-642c12dd03fd-kube-api-access-kznh6\") pod \"kube-state-metrics-56d9f944b4-jl7tq\" (UID: \"6e1d3688-4227-4f08-90ab-642c12dd03fd\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.171640 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.171611    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"
Jul 24 17:41:21.311221 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.311182    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq"]
Jul 24 17:41:21.313985 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:21.313946    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1d3688_4227_4f08_90ab_642c12dd03fd.slice/crio-244afcf2e6770631ef855b0783ae1779c8876c1d701c9f7b1623d1ebe13bc3ad WatchSource:0}: Error finding container 244afcf2e6770631ef855b0783ae1779c8876c1d701c9f7b1623d1ebe13bc3ad: Status 404 returned error can't find the container with id 244afcf2e6770631ef855b0783ae1779c8876c1d701c9f7b1623d1ebe13bc3ad
Jul 24 17:41:21.577342 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.577233    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:21.577505 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:21.577381    2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 17:41:21.577505 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:21.577469    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls podName:71cbfa0f-d3dc-43c9-9488-8a3953536703 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:22.577450615 +0000 UTC m=+280.981391830 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls") pod "node-exporter-v6jgr" (UID: "71cbfa0f-d3dc-43c9-9488-8a3953536703") : secret "node-exporter-tls" not found
Jul 24 17:41:21.962737 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.962700    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:41:21.978390 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.978359    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:21.984121 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984091    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 24 17:41:21.984321 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984091    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 24 17:41:21.984571 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984553    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 24 17:41:21.984723 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984602    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 24 17:41:21.984723 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984607    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 24 17:41:21.984723 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984696    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 24 17:41:21.984988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.984968    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-s828z\""
Jul 24 17:41:21.985086 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.985047    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 24 17:41:21.985086 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.985057    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 24 17:41:21.985086 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:21.985069    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 24 17:41:22.012430 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.012397    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:41:22.079848 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.079812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.079848 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.079857    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080110 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.079889    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080110 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.079919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-out\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080110 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080057    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080110 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-web-config\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080364 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080364 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080364 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080237    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080364 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080301    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-volume\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080563 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080563 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080405    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fz5b\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-kube-api-access-2fz5b\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.080563 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.080469    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.092586 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.091944    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq" event={"ID":"6e1d3688-4227-4f08-90ab-642c12dd03fd","Type":"ContainerStarted","Data":"244afcf2e6770631ef855b0783ae1779c8876c1d701c9f7b1623d1ebe13bc3ad"}
Jul 24 17:41:22.181301 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181251    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181309    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-out\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181345    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181366    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-web-config\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181426    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181744 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:22.181541    2575 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 24 17:41:22.181744 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:22.181619    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls podName:39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19 nodeName:}" failed. No retries permitted until 2026-07-24 17:41:22.681596781 +0000 UTC m=+281.085537991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19") : secret "alertmanager-main-tls" not found
Jul 24 17:41:22.181744 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181664    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181744 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181710    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-volume\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.181958 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.182118 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.181919    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.182228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.182129    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fz5b\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-kube-api-access-2fz5b\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.182228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.182195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.182404 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.182246    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.182404 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.182290    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.182588 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.182562    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.183436 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.182727    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.184586 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.184557    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.184791 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.184767    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-out\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.184899 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.184875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-web-config\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.185126 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.185099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-volume\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.185383 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.185360    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-tls-assets\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.185521 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.185394    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.185661 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.185640    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.186175 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.186153    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.193223 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.193197    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fz5b\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-kube-api-access-2fz5b\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.586729 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.586685    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:22.589163 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.589132    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/71cbfa0f-d3dc-43c9-9488-8a3953536703-node-exporter-tls\") pod \"node-exporter-v6jgr\" (UID: \"71cbfa0f-d3dc-43c9-9488-8a3953536703\") " pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:22.677379 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.677330    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v6jgr"
Jul 24 17:41:22.687694 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.687663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.690152 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.690116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:22.706937 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:22.706900    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71cbfa0f_d3dc_43c9_9488_8a3953536703.slice/crio-688e924f8820e2c0f94211d3797f25b43e610e50bcc40a03334f253825616169 WatchSource:0}: Error finding container 688e924f8820e2c0f94211d3797f25b43e610e50bcc40a03334f253825616169: Status 404 returned error can't find the container with id 688e924f8820e2c0f94211d3797f25b43e610e50bcc40a03334f253825616169
Jul 24 17:41:22.890906 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:22.890815    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:41:23.098593 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.098542    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6jgr" event={"ID":"71cbfa0f-d3dc-43c9-9488-8a3953536703","Type":"ContainerStarted","Data":"688e924f8820e2c0f94211d3797f25b43e610e50bcc40a03334f253825616169"}
Jul 24 17:41:23.170151 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.169796    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:41:23.173004 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:23.172966    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39e4c720_f11b_4e9c_a6b2_f19f3e0f8b19.slice/crio-2098b09d35dbd3274c12a9f28f9ef573e5a0ede2d6603929f92bc1e2d7e5338a WatchSource:0}: Error finding container 2098b09d35dbd3274c12a9f28f9ef573e5a0ede2d6603929f92bc1e2d7e5338a: Status 404 returned error can't find the container with id 2098b09d35dbd3274c12a9f28f9ef573e5a0ede2d6603929f92bc1e2d7e5338a
Jul 24 17:41:23.717336 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:23.717296    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-sfqpz" podUID="f3042e4e-4106-4f3c-be76-9677db69bbf2"
Jul 24 17:41:23.833163 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.833125    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c94485f49-5xcwq"]
Jul 24 17:41:23.836057 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.836031    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.838440 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.838412    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 24 17:41:23.838604 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.838488    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 24 17:41:23.838604 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.838522    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 24 17:41:23.838760 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.838740    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 24 17:41:23.838883 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.838864    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-qj9dd\""
Jul 24 17:41:23.839069 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.839040    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8525465ov4qdq\""
Jul 24 17:41:23.839069 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.839061    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 24 17:41:23.853853 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.853816    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c94485f49-5xcwq"]
Jul 24 17:41:23.899665 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899629    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.899863 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899697    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.899863 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899724    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr5c4\" (UniqueName: \"kubernetes.io/projected/ca9318da-5d7b-4901-83a2-d5b53607450f-kube-api-access-jr5c4\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.899863 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899773    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-grpc-tls\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.899863 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899848    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.900080 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899889    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.900080 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899916    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca9318da-5d7b-4901-83a2-d5b53607450f-metrics-client-ca\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:23.900080 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:23.899955    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-tls\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001199 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-grpc-tls\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001404 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001274    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001404 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001340    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001404 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001374    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca9318da-5d7b-4901-83a2-d5b53607450f-metrics-client-ca\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001589 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001413    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-tls\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001589 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001589 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001569    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.001740 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.001594    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr5c4\" (UniqueName: \"kubernetes.io/projected/ca9318da-5d7b-4901-83a2-d5b53607450f-kube-api-access-jr5c4\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.002364 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.002307    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca9318da-5d7b-4901-83a2-d5b53607450f-metrics-client-ca\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.005375 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.005286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-grpc-tls\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.005515 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.005411    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.010854 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.005822    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.010854 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.006734    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-tls\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.010854 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.006940    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.010854 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.007644    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ca9318da-5d7b-4901-83a2-d5b53607450f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.011515 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.011487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr5c4\" (UniqueName: \"kubernetes.io/projected/ca9318da-5d7b-4901-83a2-d5b53607450f-kube-api-access-jr5c4\") pod \"thanos-querier-5c94485f49-5xcwq\" (UID: \"ca9318da-5d7b-4901-83a2-d5b53607450f\") " pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.104838 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.104802    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq" event={"ID":"6e1d3688-4227-4f08-90ab-642c12dd03fd","Type":"ContainerStarted","Data":"c666be5830f538654da606c11983e5be75f0a1ff8ec89806213d87ae9691741c"}
Jul 24 17:41:24.105396 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.104916    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq" event={"ID":"6e1d3688-4227-4f08-90ab-642c12dd03fd","Type":"ContainerStarted","Data":"32b50b15e75822059a83c2366b2ed7811ac74a3aa56de13b5c4ae4ff268eed62"}
Jul 24 17:41:24.105396 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.104950    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq" event={"ID":"6e1d3688-4227-4f08-90ab-642c12dd03fd","Type":"ContainerStarted","Data":"0daecd15a6ddc122a909306fb3de70ba597e0ffb6fb4fc8ecfbf7c453bcbe209"}
Jul 24 17:41:24.106253 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.106222    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"2098b09d35dbd3274c12a9f28f9ef573e5a0ede2d6603929f92bc1e2d7e5338a"}
Jul 24 17:41:24.107939 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.107913    2575 generic.go:356] "Generic (PLEG): container finished" podID="71cbfa0f-d3dc-43c9-9488-8a3953536703" containerID="c7b144993855f73d491c160f112e4ba911f98adc2c251048d611f318411ebb94" exitCode=0
Jul 24 17:41:24.108046 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.107998    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6jgr" event={"ID":"71cbfa0f-d3dc-43c9-9488-8a3953536703","Type":"ContainerDied","Data":"c7b144993855f73d491c160f112e4ba911f98adc2c251048d611f318411ebb94"}
Jul 24 17:41:24.108046 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.108016    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:41:24.124602 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.124532    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-jl7tq" podStartSLOduration=2.410399402 podStartE2EDuration="4.124505459s" totalImagesPullingTime="1.714106057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:24.122610178 +0000 UTC m=+282.526551430" watchObservedRunningTime="2026-07-24 17:41:24.124505459 +0000 UTC m=+282.528446688"
Jul 24 17:41:24.164732 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.164648    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:24.429568 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:24.429533    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c94485f49-5xcwq"]
Jul 24 17:41:24.464762 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:24.464730    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9318da_5d7b_4901_83a2_d5b53607450f.slice/crio-ab5f66e09f94ddaaf335b7bce6eec08205d02be53bd0e8194777d905ae8657cc WatchSource:0}: Error finding container ab5f66e09f94ddaaf335b7bce6eec08205d02be53bd0e8194777d905ae8657cc: Status 404 returned error can't find the container with id ab5f66e09f94ddaaf335b7bce6eec08205d02be53bd0e8194777d905ae8657cc
Jul 24 17:41:25.114603 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.114565    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7" exitCode=0
Jul 24 17:41:25.115096 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.114665    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"}
Jul 24 17:41:25.117493 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.117462    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6jgr" event={"ID":"71cbfa0f-d3dc-43c9-9488-8a3953536703","Type":"ContainerStarted","Data":"b850638b83e71eca5ce8e31f0b830101adfbc4e2a37b7db88db6953fafa3a880"}
Jul 24 17:41:25.117599 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.117533    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v6jgr" event={"ID":"71cbfa0f-d3dc-43c9-9488-8a3953536703","Type":"ContainerStarted","Data":"27d0595d9e15a43b75093d6bedc5533cca22068e8a0b0728dd91292cf3cda4c2"}
Jul 24 17:41:25.119374 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.118956    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"ab5f66e09f94ddaaf335b7bce6eec08205d02be53bd0e8194777d905ae8657cc"}
Jul 24 17:41:25.155777 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.155715    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v6jgr" podStartSLOduration=4.138539731 podStartE2EDuration="5.155694442s" totalImagesPullingTime="943.263844ms" totalInitContainerRuntime="73.890867ms" isStatefulPod=true podCreationTimestamp="2026-07-24 17:41:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:25.154074823 +0000 UTC m=+283.558016084" watchObservedRunningTime="2026-07-24 17:41:25.155694442 +0000 UTC m=+283.559635671"
Jul 24 17:41:25.306834 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.306789    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6b47d57b47-hv7vd"]
Jul 24 17:41:25.309209 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.309183    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.311317 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.311294    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5ncpz\""
Jul 24 17:41:25.311597 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.311564    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-772ttn3oej3dh\""
Jul 24 17:41:25.311734 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.311639    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 24 17:41:25.311810 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.311796    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 24 17:41:25.311894 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.311877    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 24 17:41:25.311949 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.311895    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 24 17:41:25.318168 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.318137    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b47d57b47-hv7vd"]
Jul 24 17:41:25.414640 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-client-ca-bundle\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.414640 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d521f2a-e3f8-48a6-95f2-77015e928da1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.414859 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-secret-metrics-server-client-certs\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.414859 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7d521f2a-e3f8-48a6-95f2-77015e928da1-audit-log\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.414859 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414688    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-secret-metrics-server-tls\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.414994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7d521f2a-e3f8-48a6-95f2-77015e928da1-metrics-server-audit-profiles\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.414994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.414914    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44kl6\" (UniqueName: \"kubernetes.io/projected/7d521f2a-e3f8-48a6-95f2-77015e928da1-kube-api-access-44kl6\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516008 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.515973    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d521f2a-e3f8-48a6-95f2-77015e928da1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516190 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516017    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-secret-metrics-server-client-certs\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516190 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516043    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7d521f2a-e3f8-48a6-95f2-77015e928da1-audit-log\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516190 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516071    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-secret-metrics-server-tls\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516190 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7d521f2a-e3f8-48a6-95f2-77015e928da1-metrics-server-audit-profiles\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516190 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44kl6\" (UniqueName: \"kubernetes.io/projected/7d521f2a-e3f8-48a6-95f2-77015e928da1-kube-api-access-44kl6\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516460 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516218    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-client-ca-bundle\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.516517 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.516476    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7d521f2a-e3f8-48a6-95f2-77015e928da1-audit-log\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.517233 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.517201    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d521f2a-e3f8-48a6-95f2-77015e928da1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.517456 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.517330    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7d521f2a-e3f8-48a6-95f2-77015e928da1-metrics-server-audit-profiles\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.519373 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.519317    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-secret-metrics-server-client-certs\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.519507 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.519489    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-secret-metrics-server-tls\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.520775 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.520751    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d521f2a-e3f8-48a6-95f2-77015e928da1-client-ca-bundle\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.528023 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.527954    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44kl6\" (UniqueName: \"kubernetes.io/projected/7d521f2a-e3f8-48a6-95f2-77015e928da1-kube-api-access-44kl6\") pod \"metrics-server-6b47d57b47-hv7vd\" (UID: \"7d521f2a-e3f8-48a6-95f2-77015e928da1\") " pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.566691 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.566137    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"]
Jul 24 17:41:25.568805 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.568775    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:25.571132 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.571044    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-csjf4\""
Jul 24 17:41:25.571470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.571448    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 24 17:41:25.577382 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.577350    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"]
Jul 24 17:41:25.617788 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.617667    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:25.617788 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.617719    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd94v\" (UniqueName: \"kubernetes.io/projected/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-kube-api-access-cd94v\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:25.622729 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.622561    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:25.718530 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.718488    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:25.718737 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.718538    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd94v\" (UniqueName: \"kubernetes.io/projected/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-kube-api-access-cd94v\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:25.718737 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:25.718676    2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 24 17:41:25.718858 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:25.718763    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-monitoring-plugin-cert podName:ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d nodeName:}" failed. No retries permitted until 2026-07-24 17:41:26.218740936 +0000 UTC m=+284.622682147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-2sqqw" (UID: "ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d") : secret "monitoring-plugin-cert" not found
Jul 24 17:41:25.727633 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:25.727588    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd94v\" (UniqueName: \"kubernetes.io/projected/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-kube-api-access-cd94v\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:26.224393 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:26.224019    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:26.235324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:26.235238    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-2sqqw\" (UID: \"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:26.483082 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:26.482986    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:26.676570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:26.676523    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6b47d57b47-hv7vd"]
Jul 24 17:41:26.681902 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:26.681868    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d521f2a_e3f8_48a6_95f2_77015e928da1.slice/crio-534171fd30e77f46df4b2f32ae63f81d52a2250769422420b394c6f7607ba303 WatchSource:0}: Error finding container 534171fd30e77f46df4b2f32ae63f81d52a2250769422420b394c6f7607ba303: Status 404 returned error can't find the container with id 534171fd30e77f46df4b2f32ae63f81d52a2250769422420b394c6f7607ba303
Jul 24 17:41:26.688464 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:26.688430    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"]
Jul 24 17:41:26.693399 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:26.693370    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4e5bd8_4f8c_40b6_8884_82d0d0b6eb1d.slice/crio-27e1d92b72f2740f45f5df001e98975e571420ffa4a00c878f65bff9b2f2628e WatchSource:0}: Error finding container 27e1d92b72f2740f45f5df001e98975e571420ffa4a00c878f65bff9b2f2628e: Status 404 returned error can't find the container with id 27e1d92b72f2740f45f5df001e98975e571420ffa4a00c878f65bff9b2f2628e
Jul 24 17:41:27.020065 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.019978    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:41:27.025353 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.025318    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.027788 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.027749    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 17:41:27.028006 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.027987    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028457    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028540    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cfkf4rkg5dl8a\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028552    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028601    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028619    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028540    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vk7mt\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028545    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 17:41:27.028736 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028555    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 17:41:27.029310 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028756    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 17:41:27.029310 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.028807    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 17:41:27.029310 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.029048    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 17:41:27.029310 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.029110    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 17:41:27.029310 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.029236    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 17:41:27.032890 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.032688    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 17:41:27.042342 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.042310    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:41:27.134382 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134330    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"}
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134422    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"}
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134461    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"}
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134485    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134495    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"}
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134523    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134551    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"}
Jul 24 17:41:27.134570 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134553    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134583    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134691    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134743    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134831    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.134873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134858    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134897    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8dcn\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-kube-api-access-v8dcn\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.134978    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135009    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135062    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135213    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135283    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135324    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135379    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.135454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.135409    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.139505 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.139463    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"9a09f0b5a1aa17334674b43556dddbcb2bd69fe10cb1ae1c3443682af1044ed7"}
Jul 24 17:41:27.139694 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.139555    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"785e9a269e283d29888650090b0df99f546261f0d8b738ecd7de4a82050eabbe"}
Jul 24 17:41:27.139694 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.139606    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"a0d53451057fdcde6141e7b5e7d5c6e8abe9b31985bd7b211ee8d25f3d240060"}
Jul 24 17:41:27.141041 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.141009    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw" event={"ID":"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d","Type":"ContainerStarted","Data":"27e1d92b72f2740f45f5df001e98975e571420ffa4a00c878f65bff9b2f2628e"}
Jul 24 17:41:27.144438 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.144398    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd" event={"ID":"7d521f2a-e3f8-48a6-95f2-77015e928da1","Type":"ContainerStarted","Data":"534171fd30e77f46df4b2f32ae63f81d52a2250769422420b394c6f7607ba303"}
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.239967    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240025    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240055    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240173    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240199    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240232    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240297    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240387    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240451    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.240469 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240484    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240509    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240537    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8dcn\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-kube-api-access-v8dcn\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240658    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240717    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240727    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.241549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.240742    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.243908 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.243875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.245822 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.244570    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.245822 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.244609    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.245822 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.244802    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config-out\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.245822 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.245479    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.245822 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.245496    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.246695 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.246339    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.247322 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.247297    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.252655 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.252598    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.252655 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.252646    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.256517 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.255778    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.256517 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.255875    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-web-config\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.257936 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.257259    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.257936 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.257334    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.257936 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.257689    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.257936 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.257876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.258803 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.258524    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.258803 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.258698    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.260324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.260279    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8dcn\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-kube-api-access-v8dcn\") pod \"prometheus-k8s-0\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.342002 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.341838    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:27.443626 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.443051    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:41:27.446883 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.446807    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f3042e4e-4106-4f3c-be76-9677db69bbf2-metrics-tls\") pod \"dns-default-sfqpz\" (UID: \"f3042e4e-4106-4f3c-be76-9677db69bbf2\") " pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:41:27.571341 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.571211    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:41:27.617745 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:27.617661    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e25139_0fd7_4a21_a4c3_0a796262e8b7.slice/crio-5d77df9aa4604573a1ca152e5b363a0036ca1c61e12e1a042d11b98008c226c9 WatchSource:0}: Error finding container 5d77df9aa4604573a1ca152e5b363a0036ca1c61e12e1a042d11b98008c226c9: Status 404 returned error can't find the container with id 5d77df9aa4604573a1ca152e5b363a0036ca1c61e12e1a042d11b98008c226c9
Jul 24 17:41:27.710949 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.710926    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7mn5h\""
Jul 24 17:41:27.719541 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:27.719514    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:41:28.152761 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:28.152713    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"4b0393c8612b181962ad606c8093af304104e3d85ae5657c2c939782852501f4"}
Jul 24 17:41:28.154500 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:28.154466    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b" exitCode=0
Jul 24 17:41:28.154645 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:28.154511    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"}
Jul 24 17:41:28.154645 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:28.154574    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"5d77df9aa4604573a1ca152e5b363a0036ca1c61e12e1a042d11b98008c226c9"}
Jul 24 17:41:28.679972 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:28.679941    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-sfqpz"]
Jul 24 17:41:28.682481 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:41:28.682444    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3042e4e_4106_4f3c_be76_9677db69bbf2.slice/crio-d5e913c9c333e24b84280f125bc731e7ffe965540d8dccb99596121a283dcf6f WatchSource:0}: Error finding container d5e913c9c333e24b84280f125bc731e7ffe965540d8dccb99596121a283dcf6f: Status 404 returned error can't find the container with id d5e913c9c333e24b84280f125bc731e7ffe965540d8dccb99596121a283dcf6f
Jul 24 17:41:29.164763 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.164725    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"3727f8f9e6910f1b48ba5af1cb6fec7a56a024af2c7a136402cb6756546530cc"}
Jul 24 17:41:29.164946 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.164813    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" event={"ID":"ca9318da-5d7b-4901-83a2-d5b53607450f","Type":"ContainerStarted","Data":"3369c1af34f0f07514ad9dc954d4079c29daeec44f83b3985760bbe3cf0b8c96"}
Jul 24 17:41:29.165483 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.165459    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:29.168052 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.168030    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd" event={"ID":"7d521f2a-e3f8-48a6-95f2-77015e928da1","Type":"ContainerStarted","Data":"0a10f1256c9a9fb92b8a87ec1fd7d1d418b6f020740aa93084bb3997a136fc67"}
Jul 24 17:41:29.172634 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.172603    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerStarted","Data":"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"}
Jul 24 17:41:29.175396 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.175367    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw" event={"ID":"ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d","Type":"ContainerStarted","Data":"a9211efe2fabb940647dfaa24fd26a27ab284345e37e171ec87abc6da4ea3ce6"}
Jul 24 17:41:29.175615 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.175594    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:29.176972 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.176936    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sfqpz" event={"ID":"f3042e4e-4106-4f3c-be76-9677db69bbf2","Type":"ContainerStarted","Data":"d5e913c9c333e24b84280f125bc731e7ffe965540d8dccb99596121a283dcf6f"}
Jul 24 17:41:29.182749 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.182727    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw"
Jul 24 17:41:29.190521 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.190464    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq" podStartSLOduration=3.245174905 podStartE2EDuration="6.190440301s" totalImagesPullingTime="2.945265396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:29.18750013 +0000 UTC m=+287.591441376" watchObservedRunningTime="2026-07-24 17:41:29.190440301 +0000 UTC m=+287.594381531"
Jul 24 17:41:29.203727 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.203312    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-2sqqw" podStartSLOduration=2.379217795 podStartE2EDuration="4.203290697s" totalImagesPullingTime="1.824072902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:29.201799239 +0000 UTC m=+287.605740488" watchObservedRunningTime="2026-07-24 17:41:29.203290697 +0000 UTC m=+287.607231928"
Jul 24 17:41:29.227887 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:29.227771    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.899411177 podStartE2EDuration="8.227750757s" totalImagesPullingTime="3.296614506s" totalInitContainerRuntime="31.725074ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:29.226700881 +0000 UTC m=+287.630642163" watchObservedRunningTime="2026-07-24 17:41:29.227750757 +0000 UTC m=+287.631691987"
Jul 24 17:41:32.195798 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.195758    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"}
Jul 24 17:41:32.196308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.195839    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"}
Jul 24 17:41:32.196308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.195879    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"}
Jul 24 17:41:32.196308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.195907    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"}
Jul 24 17:41:32.196308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.195938    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"}
Jul 24 17:41:32.196308 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.195961    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerStarted","Data":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"}
Jul 24 17:41:32.197646 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.197625    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sfqpz" event={"ID":"f3042e4e-4106-4f3c-be76-9677db69bbf2","Type":"ContainerStarted","Data":"3d578c921d3ab8db486c87761702e8db9d312ede49efab21fea55a1a1996d31a"}
Jul 24 17:41:32.197766 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.197664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-sfqpz" event={"ID":"f3042e4e-4106-4f3c-be76-9677db69bbf2","Type":"ContainerStarted","Data":"a231f1cc62546146140d9c9dd86394e0dbef5b75c73b808b10eab6453ea895c4"}
Jul 24 17:41:32.197766 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.197684    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:41:32.225019 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.224952    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd" podStartSLOduration=5.3925909 podStartE2EDuration="7.224931077s" totalImagesPullingTime="1.832340177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:29.247552093 +0000 UTC m=+287.651493351" watchObservedRunningTime="2026-07-24 17:41:32.224931077 +0000 UTC m=+290.628872311"
Jul 24 17:41:32.226679 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.226626    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.932514159 podStartE2EDuration="6.226609378s" totalImagesPullingTime="3.269905626s" totalInitContainerRuntime="24.189593ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:32.223576856 +0000 UTC m=+290.627518108" watchObservedRunningTime="2026-07-24 17:41:32.226609378 +0000 UTC m=+290.630550608"
Jul 24 17:41:32.243568 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.243511    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-sfqpz" podStartSLOduration=252.507952562 podStartE2EDuration="4m15.243489445s" totalImagesPullingTime="2.735536883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:41:32.242338619 +0000 UTC m=+290.646279874" watchObservedRunningTime="2026-07-24 17:41:32.243489445 +0000 UTC m=+290.647430675"
Jul 24 17:41:32.342381 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:32.342285    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:41:35.133936 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:41:35.133891    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-x2cmc" podUID="cbeb29f5-efec-44dc-aed1-21dc7e02dca4"
Jul 24 17:41:35.187970 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:35.187941    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c94485f49-5xcwq"
Jul 24 17:41:42.022905 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:42.022875    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:41:42.023425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:42.023066    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:41:42.032510 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:42.032486    2575 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 17:41:42.207581 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:42.207554    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-sfqpz"
Jul 24 17:41:45.623678 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:45.623636    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:45.624110 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:45.623702    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:41:47.132200 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:41:47.132153    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:42:05.628678 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:05.628650    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:42:05.632682 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:05.632658    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6b47d57b47-hv7vd"
Jul 24 17:42:09.359487 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:09.359392    2575 generic.go:356] "Generic (PLEG): container finished" podID="663e6631-6566-4b7b-9518-da24c85fc15b" containerID="dd5a11bab32bfa5fb42fea50d3123fac0c11aba6cc6d5a3054c06af24a711365" exitCode=0
Jul 24 17:42:09.359487 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:09.359469    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k" event={"ID":"663e6631-6566-4b7b-9518-da24c85fc15b","Type":"ContainerDied","Data":"dd5a11bab32bfa5fb42fea50d3123fac0c11aba6cc6d5a3054c06af24a711365"}
Jul 24 17:42:09.359926 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:09.359764    2575 scope.go:117] "RemoveContainer" containerID="dd5a11bab32bfa5fb42fea50d3123fac0c11aba6cc6d5a3054c06af24a711365"
Jul 24 17:42:10.372072 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:10.372023    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-47r7k" event={"ID":"663e6631-6566-4b7b-9518-da24c85fc15b","Type":"ContainerStarted","Data":"a6cff860bc9e7386c849813459d05cc54570b39b99a20813c5f4199dc85c5b56"}
Jul 24 17:42:24.428855 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:24.428819    2575 generic.go:356] "Generic (PLEG): container finished" podID="970c6912-05fb-4fd7-a428-407244b542fc" containerID="6945ce15875012908f61c874cc8b15c4f0b5aad8b120642b800f920543d0c510" exitCode=0
Jul 24 17:42:24.429309 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:24.428876    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-rcfqt" event={"ID":"970c6912-05fb-4fd7-a428-407244b542fc","Type":"ContainerDied","Data":"6945ce15875012908f61c874cc8b15c4f0b5aad8b120642b800f920543d0c510"}
Jul 24 17:42:24.429309 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:24.429199    2575 scope.go:117] "RemoveContainer" containerID="6945ce15875012908f61c874cc8b15c4f0b5aad8b120642b800f920543d0c510"
Jul 24 17:42:25.434356 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:25.434322    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-rcfqt" event={"ID":"970c6912-05fb-4fd7-a428-407244b542fc","Type":"ContainerStarted","Data":"93c63c5510277e29a893aa0798a8e1e0b4b0df6ba52e382bfb3a23de5d0735af"}
Jul 24 17:42:27.342708 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:27.342662    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:27.359105 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:27.359076    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:27.459968 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:27.459941    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:41.122200 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122112    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:42:41.122755 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122449    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="alertmanager" containerID="cri-o://515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee" gracePeriod=120
Jul 24 17:42:41.122755 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122504    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-metric" containerID="cri-o://ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde" gracePeriod=120
Jul 24 17:42:41.122755 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122542    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="prom-label-proxy" containerID="cri-o://06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850" gracePeriod=120
Jul 24 17:42:41.122755 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122562    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-web" containerID="cri-o://f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d" gracePeriod=120
Jul 24 17:42:41.122755 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122588    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy" containerID="cri-o://63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044" gracePeriod=120
Jul 24 17:42:41.122755 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.122612    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="config-reloader" containerID="cri-o://0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c" gracePeriod=120
Jul 24 17:42:41.510612 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510581    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850" exitCode=0
Jul 24 17:42:41.510612 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510606    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044" exitCode=0
Jul 24 17:42:41.510612 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510612    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c" exitCode=0
Jul 24 17:42:41.510612 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510617    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee" exitCode=0
Jul 24 17:42:41.510941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510655    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"}
Jul 24 17:42:41.510941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510728    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"}
Jul 24 17:42:41.510941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510757    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"}
Jul 24 17:42:41.510941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:41.510783    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"}
Jul 24 17:42:42.380047 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.380019    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:42:42.456015 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.455921    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456015 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.455963    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fz5b\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-kube-api-access-2fz5b\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456015 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.455988    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-web\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456024    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-web-config\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456081    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-volume\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456131    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-tls-assets\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456164    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-cluster-tls-config\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456192    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-metric\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456219    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-out\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456243    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456325 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456295    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-trusted-ca-bundle\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456345    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-main-db\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.456712 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.456367    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-metrics-client-ca\") pod \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\" (UID: \"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19\") "
Jul 24 17:42:42.457435 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.457093    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:42.457800 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.457742    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:42.458252 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.458111    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:42.460807 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.460744    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.461451 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.461403    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-kube-api-access-2fz5b" (OuterVolumeSpecName: "kube-api-access-2fz5b") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "kube-api-access-2fz5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:42.461727 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.461695    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-out" (OuterVolumeSpecName: "config-out") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:42.461883 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.461858    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-volume" (OuterVolumeSpecName: "config-volume") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.462068 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.462031    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.462068 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.462041    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:42.462410 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.462385    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.462479 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.462413    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.468386 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.468350    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.474146 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.474113    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-web-config" (OuterVolumeSpecName: "web-config") pod "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" (UID: "39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:42.518751 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518717    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde" exitCode=0
Jul 24 17:42:42.518751 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518744    2575 generic.go:356] "Generic (PLEG): container finished" podID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerID="f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d" exitCode=0
Jul 24 17:42:42.519021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518812    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 24 17:42:42.519021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518811    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"}
Jul 24 17:42:42.519021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518907    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"}
Jul 24 17:42:42.519021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518953    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19","Type":"ContainerDied","Data":"2098b09d35dbd3274c12a9f28f9ef573e5a0ede2d6603929f92bc1e2d7e5338a"}
Jul 24 17:42:42.519021 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.518972    2575 scope.go:117] "RemoveContainer" containerID="06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"
Jul 24 17:42:42.527462 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.527439    2575 scope.go:117] "RemoveContainer" containerID="ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"
Jul 24 17:42:42.535166 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.535144    2575 scope.go:117] "RemoveContainer" containerID="63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"
Jul 24 17:42:42.543413 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.543378    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:42:42.544451 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.544324    2575 scope.go:117] "RemoveContainer" containerID="f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"
Jul 24 17:42:42.547124 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.547103    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 24 17:42:42.552870 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.552850    2575 scope.go:117] "RemoveContainer" containerID="0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"
Jul 24 17:42:42.557379 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557351    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-tls-assets\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557379 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557377    2575 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-cluster-tls-config\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557387    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557397    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-out\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557405    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-main-tls\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557414    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557422    2575 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-alertmanager-main-db\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557431    2575 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-metrics-client-ca\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557439    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557448    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fz5b\" (UniqueName: \"kubernetes.io/projected/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-kube-api-access-2fz5b\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557456    2575 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557465    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-web-config\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.557549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.557473    2575 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19-config-volume\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:42.561126 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.561105    2575 scope.go:117] "RemoveContainer" containerID="515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"
Jul 24 17:42:42.570120 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.570098    2575 scope.go:117] "RemoveContainer" containerID="6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"
Jul 24 17:42:42.578553 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.578534    2575 scope.go:117] "RemoveContainer" containerID="06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"
Jul 24 17:42:42.578902 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.578878    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850\": container with ID starting with 06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850 not found: ID does not exist" containerID="06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"
Jul 24 17:42:42.578952 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.578913    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"} err="failed to get container status \"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850\": rpc error: code = NotFound desc = could not find container \"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850\": container with ID starting with 06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850 not found: ID does not exist"
Jul 24 17:42:42.578952 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.578950    2575 scope.go:117] "RemoveContainer" containerID="ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"
Jul 24 17:42:42.579255 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.579218    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde\": container with ID starting with ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde not found: ID does not exist" containerID="ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"
Jul 24 17:42:42.579322 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.579274    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"} err="failed to get container status \"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde\": rpc error: code = NotFound desc = could not find container \"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde\": container with ID starting with ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde not found: ID does not exist"
Jul 24 17:42:42.579322 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.579292    2575 scope.go:117] "RemoveContainer" containerID="63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"
Jul 24 17:42:42.579538 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.579520    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044\": container with ID starting with 63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044 not found: ID does not exist" containerID="63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"
Jul 24 17:42:42.579574 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.579544    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"} err="failed to get container status \"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044\": rpc error: code = NotFound desc = could not find container \"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044\": container with ID starting with 63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044 not found: ID does not exist"
Jul 24 17:42:42.579574 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.579559    2575 scope.go:117] "RemoveContainer" containerID="f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"
Jul 24 17:42:42.579849 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.579830    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d\": container with ID starting with f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d not found: ID does not exist" containerID="f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"
Jul 24 17:42:42.579924 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.579849    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"} err="failed to get container status \"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d\": rpc error: code = NotFound desc = could not find container \"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d\": container with ID starting with f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d not found: ID does not exist"
Jul 24 17:42:42.579924 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.579866    2575 scope.go:117] "RemoveContainer" containerID="0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"
Jul 24 17:42:42.580175 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.580154    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c\": container with ID starting with 0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c not found: ID does not exist" containerID="0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"
Jul 24 17:42:42.580233 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580174    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"} err="failed to get container status \"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c\": rpc error: code = NotFound desc = could not find container \"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c\": container with ID starting with 0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c not found: ID does not exist"
Jul 24 17:42:42.580233 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580196    2575 scope.go:117] "RemoveContainer" containerID="515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"
Jul 24 17:42:42.580422 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.580407    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee\": container with ID starting with 515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee not found: ID does not exist" containerID="515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"
Jul 24 17:42:42.580466 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580434    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"} err="failed to get container status \"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee\": rpc error: code = NotFound desc = could not find container \"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee\": container with ID starting with 515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee not found: ID does not exist"
Jul 24 17:42:42.580466 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580446    2575 scope.go:117] "RemoveContainer" containerID="6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"
Jul 24 17:42:42.580736 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:42.580718    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7\": container with ID starting with 6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7 not found: ID does not exist" containerID="6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"
Jul 24 17:42:42.580805 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580740    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"} err="failed to get container status \"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7\": rpc error: code = NotFound desc = could not find container \"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7\": container with ID starting with 6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7 not found: ID does not exist"
Jul 24 17:42:42.580805 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580758    2575 scope.go:117] "RemoveContainer" containerID="06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"
Jul 24 17:42:42.581001 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.580986    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850"} err="failed to get container status \"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850\": rpc error: code = NotFound desc = could not find container \"06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850\": container with ID starting with 06d76b5f3249f9ebd60b3b207ba9a63f1666b88c499869cb05f1321f83b2b850 not found: ID does not exist"
Jul 24 17:42:42.581059 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581001    2575 scope.go:117] "RemoveContainer" containerID="ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"
Jul 24 17:42:42.581231 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581214    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde"} err="failed to get container status \"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde\": rpc error: code = NotFound desc = could not find container \"ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde\": container with ID starting with ea31644094b5acf1cb8a814aacbc109b9c05f638b8981b2c31fe09064169ffde not found: ID does not exist"
Jul 24 17:42:42.581303 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581243    2575 scope.go:117] "RemoveContainer" containerID="63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"
Jul 24 17:42:42.581497 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581478    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044"} err="failed to get container status \"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044\": rpc error: code = NotFound desc = could not find container \"63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044\": container with ID starting with 63c20ef550278764998276b27d34056ae21cd6450a81559190f5c8ed378ef044 not found: ID does not exist"
Jul 24 17:42:42.581497 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581497    2575 scope.go:117] "RemoveContainer" containerID="f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"
Jul 24 17:42:42.581764 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581728    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d"} err="failed to get container status \"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d\": rpc error: code = NotFound desc = could not find container \"f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d\": container with ID starting with f6eec18d6027f9df42936c2b8fed4ddad4653aaa95d78b7e5064863b42c20b5d not found: ID does not exist"
Jul 24 17:42:42.581764 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581763    2575 scope.go:117] "RemoveContainer" containerID="0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"
Jul 24 17:42:42.582007 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.581990    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c"} err="failed to get container status \"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c\": rpc error: code = NotFound desc = could not find container \"0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c\": container with ID starting with 0f3b59e7d875382bf3da69f796f82fde8ce8be94710d337618939d8742ddcd2c not found: ID does not exist"
Jul 24 17:42:42.582084 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.582009    2575 scope.go:117] "RemoveContainer" containerID="515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"
Jul 24 17:42:42.582239 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.582220    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee"} err="failed to get container status \"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee\": rpc error: code = NotFound desc = could not find container \"515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee\": container with ID starting with 515e8d4571ddce53f4c36304523328c6325d94c014f1d8a7c593ad9c840f7fee not found: ID does not exist"
Jul 24 17:42:42.582239 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.582238    2575 scope.go:117] "RemoveContainer" containerID="6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"
Jul 24 17:42:42.582492 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:42.582473    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7"} err="failed to get container status \"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7\": rpc error: code = NotFound desc = could not find container \"6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7\": container with ID starting with 6deb7eb493dcbb857d99884d7e54c98b3de5d03ffb575b05a56691c6b73405b7 not found: ID does not exist"
Jul 24 17:42:44.137884 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:44.137847    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" path="/var/lib/kubelet/pods/39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19/volumes"
Jul 24 17:42:45.447703 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.447667    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:42:45.448158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.447955    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="prometheus" containerID="cri-o://c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52" gracePeriod=600
Jul 24 17:42:45.448158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.447976    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="thanos-sidecar" containerID="cri-o://4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4" gracePeriod=600
Jul 24 17:42:45.448158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.447975    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy" containerID="cri-o://c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d" gracePeriod=600
Jul 24 17:42:45.448158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.448008    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="config-reloader" containerID="cri-o://31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938" gracePeriod=600
Jul 24 17:42:45.448158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.448043    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-web" containerID="cri-o://6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd" gracePeriod=600
Jul 24 17:42:45.448158 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.448137    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992" gracePeriod=600
Jul 24 17:42:45.713938 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.713911    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:45.787734 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.787705    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-tls-assets\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.787734 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.787742    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.787995 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.787767    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-web-config\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.787995 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.787791    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.787995 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.787817    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-grpc-tls\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788144 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788061    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-1\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788144 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788113    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-0\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788249 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788182    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-trusted-ca-bundle\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788336 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788251    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-metrics-client-certs\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788336 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788311    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-thanos-prometheus-http-client-file\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788445 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788353    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788445 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788387    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-kubelet-serving-ca-bundle\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788445 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788413    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-metrics-client-ca\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788445 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788409    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788453    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-kube-rbac-proxy\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788486    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config-out\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788528    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8dcn\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-kube-api-access-v8dcn\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788555    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-serving-certs-ca-bundle\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788585    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-db\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788610    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-tls\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788641    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-2\") pod \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\" (UID: \"60e25139-0fd7-4a21-a4c3-0a796262e8b7\") "
Jul 24 17:42:45.788988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.788967    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.789465 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.789298    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.790735 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.790150    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.791534 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.791469    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.791657 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.791643    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:45.791726 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.791691    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.791998 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.791968    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.792170 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.792134    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.792258 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.792205    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.792369 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.792345    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 17:42:45.792549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.792526    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.792887 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.792858    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:45.793180 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.793153    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config-out" (OuterVolumeSpecName: "config-out") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 17:42:45.793725 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.793593    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.793725 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.793687    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-kube-api-access-v8dcn" (OuterVolumeSpecName: "kube-api-access-v8dcn") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "kube-api-access-v8dcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 17:42:45.793725 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.793716    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.794324 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.794300    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.794494 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.794473    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.794602 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.794580    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config" (OuterVolumeSpecName: "config") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.804438 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.804400    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-web-config" (OuterVolumeSpecName: "web-config") pod "60e25139-0fd7-4a21-a4c3-0a796262e8b7" (UID: "60e25139-0fd7-4a21-a4c3-0a796262e8b7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 17:42:45.889919 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889861    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.889919 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889908    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.889919 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889922    2575 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-metrics-client-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.889919 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889935    2575 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889948    2575 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889960    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889974    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-metrics-client-ca\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889986    2575 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-kube-rbac-proxy\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.889997    2575 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-config-out\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890008    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8dcn\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-kube-api-access-v8dcn\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890019    2575 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890031    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-db\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890043    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890055    2575 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/60e25139-0fd7-4a21-a4c3-0a796262e8b7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890066    2575 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60e25139-0fd7-4a21-a4c3-0a796262e8b7-tls-assets\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890077    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890089    2575 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-web-config\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890101    2575 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:45.890237 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:45.890113    2575 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60e25139-0fd7-4a21-a4c3-0a796262e8b7-secret-grpc-tls\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 17:42:46.541349 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541315    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992" exitCode=0
Jul 24 17:42:46.541349 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541342    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d" exitCode=0
Jul 24 17:42:46.541349 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541351    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd" exitCode=0
Jul 24 17:42:46.541349 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541358    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4" exitCode=0
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541366    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938" exitCode=0
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541373    2575 generic.go:356] "Generic (PLEG): container finished" podID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52" exitCode=0
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541392    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541401    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"}
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541470    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"}
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541498    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"}
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541523    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"}
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541543    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541553    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"}
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541578    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"}
Jul 24 17:42:46.541933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.541609    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"60e25139-0fd7-4a21-a4c3-0a796262e8b7","Type":"ContainerDied","Data":"5d77df9aa4604573a1ca152e5b363a0036ca1c61e12e1a042d11b98008c226c9"}
Jul 24 17:42:46.554389 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.554366    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.563234 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.563201    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.563749 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.563723    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:42:46.567209 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.567184    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:42:46.571784 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.571757    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.579380 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.579352    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.587198 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.587173    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.594304 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594255    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:42:46.594718 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594685    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-web"
Jul 24 17:42:46.594718 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594716    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-web"
Jul 24 17:42:46.594718 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594725    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594731    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594737    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="init-config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594743    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="init-config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594749    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594754    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594763    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594770    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594780    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-web"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594785    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-web"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594794    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="alertmanager"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594799    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="alertmanager"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594805    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="prometheus"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594811    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="prometheus"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594817    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="thanos-sidecar"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594822    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="thanos-sidecar"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594831    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594837    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594850    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-metric"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594855    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-metric"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594863    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="init-config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594867    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="init-config-reloader"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594873    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="prom-label-proxy"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594879    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="prom-label-proxy"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594890    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-thanos"
Jul 24 17:42:46.594907 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594895    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-thanos"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594946    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594955    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="prometheus"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594960    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-thanos"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594965    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="kube-rbac-proxy-web"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594971    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-metric"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594976    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="config-reloader"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594980    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" containerName="thanos-sidecar"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594985    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="config-reloader"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594990    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy-web"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.594999    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="alertmanager"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.595006    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="prom-label-proxy"
Jul 24 17:42:46.595844 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.595012    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="39e4c720-f11b-4e9c-a6b2-f19f3e0f8b19" containerName="kube-rbac-proxy"
Jul 24 17:42:46.596374 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.596062    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.600936 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.600908    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.603069 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603046    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 24 17:42:46.603069 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603064    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 24 17:42:46.603498 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603479    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 24 17:42:46.603794 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603649    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 24 17:42:46.603794 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603688    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cfkf4rkg5dl8a\""
Jul 24 17:42:46.603794 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603702    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 24 17:42:46.603794 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.603770    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 24 17:42:46.604185 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604002    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 24 17:42:46.604185 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604074    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-vk7mt\""
Jul 24 17:42:46.604185 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604183    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 24 17:42:46.604410 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604224    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 24 17:42:46.604410 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604300    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 24 17:42:46.604410 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604283    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 24 17:42:46.604626 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.604563    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 24 17:42:46.606448 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.606429    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 24 17:42:46.607217 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.607203    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.607555 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.607536    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.607626 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.607563    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"} err="failed to get container status \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist"
Jul 24 17:42:46.607626 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.607581    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.607990 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.607858    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.607990 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.607892    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"} err="failed to get container status \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist"
Jul 24 17:42:46.607990 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.607914    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.608199 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.608176    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.608257 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.608205    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"} err="failed to get container status \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist"
Jul 24 17:42:46.608257 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.608225    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.608615 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.608595    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.608700 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.608622    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"} err="failed to get container status \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist"
Jul 24 17:42:46.608700 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.608650    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.608950 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.608927    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.609055 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.608952    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"} err="failed to get container status \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist"
Jul 24 17:42:46.609055 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.608967    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.609343 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.609283    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.609343 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.609311    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"} err="failed to get container status \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist"
Jul 24 17:42:46.609457 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.609343    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.609725 ip-10-0-140-161 kubenswrapper[2575]: E0724 17:42:46.609706    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.609810 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.609732    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"} err="failed to get container status \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist"
Jul 24 17:42:46.609810 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.609752    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.609810 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.609774    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 24 17:42:46.610048 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610026    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"} err="failed to get container status \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist"
Jul 24 17:42:46.610096 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610050    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.610327 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610307    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"} err="failed to get container status \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist"
Jul 24 17:42:46.610327 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610326    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.610590 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610574    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"} err="failed to get container status \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist"
Jul 24 17:42:46.610628 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610591    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.610829 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610808    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"} err="failed to get container status \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist"
Jul 24 17:42:46.610895 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.610829    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.611055 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611036    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"} err="failed to get container status \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist"
Jul 24 17:42:46.611119 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611057    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.611301 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611276    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"} err="failed to get container status \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist"
Jul 24 17:42:46.611374 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611301    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.611549 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611534    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"} err="failed to get container status \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist"
Jul 24 17:42:46.611642 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611549    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.611777 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611758    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"} err="failed to get container status \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist"
Jul 24 17:42:46.611825 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611778    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.611984 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611966    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"} err="failed to get container status \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist"
Jul 24 17:42:46.612050 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.611987    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.612225 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.612206    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"} err="failed to get container status \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist"
Jul 24 17:42:46.612301 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.612226    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.612470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.612454    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"} err="failed to get container status \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist"
Jul 24 17:42:46.612521 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.612470    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.612758 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.612704    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"} err="failed to get container status \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist"
Jul 24 17:42:46.612826 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.612761    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.613278 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.613195    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"} err="failed to get container status \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist"
Jul 24 17:42:46.613367 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.613257    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.613608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.613588    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"} err="failed to get container status \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist"
Jul 24 17:42:46.613608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.613607    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.614100 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614053    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"} err="failed to get container status \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist"
Jul 24 17:42:46.614100 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614073    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.614360 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614339    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"} err="failed to get container status \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist"
Jul 24 17:42:46.614418 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614361    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.614631 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614613    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"} err="failed to get container status \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist"
Jul 24 17:42:46.614631 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614630    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.614926 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614899    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"} err="failed to get container status \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist"
Jul 24 17:42:46.615006 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.614927    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.615133 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615114    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:42:46.615208 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615151    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"} err="failed to get container status \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist"
Jul 24 17:42:46.615208 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615169    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.615458 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615441    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"} err="failed to get container status \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist"
Jul 24 17:42:46.615506 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615459    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.615672 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615653    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"} err="failed to get container status \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist"
Jul 24 17:42:46.615672 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615672    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.615868 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615852    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"} err="failed to get container status \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist"
Jul 24 17:42:46.615911 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.615869    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.616062 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616047    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"} err="failed to get container status \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist"
Jul 24 17:42:46.616103 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616062    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.616273 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616245    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"} err="failed to get container status \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist"
Jul 24 17:42:46.616329 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616260    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.616484 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616468    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"} err="failed to get container status \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist"
Jul 24 17:42:46.616484 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616483    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.616699 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616684    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"} err="failed to get container status \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist"
Jul 24 17:42:46.616699 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616700    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.616967 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616931    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"} err="failed to get container status \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist"
Jul 24 17:42:46.616967 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.616961    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.617454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.617247    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"} err="failed to get container status \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist"
Jul 24 17:42:46.617454 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.617319    2575 scope.go:117] "RemoveContainer" containerID="b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"
Jul 24 17:42:46.617646 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.617622    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992"} err="failed to get container status \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": rpc error: code = NotFound desc = could not find container \"b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992\": container with ID starting with b5ae27f0b41e2fccc7af3164a26caefdcffe01c7e34be783a429cfffdc819992 not found: ID does not exist"
Jul 24 17:42:46.617646 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.617642    2575 scope.go:117] "RemoveContainer" containerID="c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"
Jul 24 17:42:46.617890 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.617871    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d"} err="failed to get container status \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": rpc error: code = NotFound desc = could not find container \"c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d\": container with ID starting with c2abd87a935bd2ed2f937defa82e1920de89f1385c44d8a1e2c612182732231d not found: ID does not exist"
Jul 24 17:42:46.617890 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.617889    2575 scope.go:117] "RemoveContainer" containerID="6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"
Jul 24 17:42:46.618146 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618122    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd"} err="failed to get container status \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": rpc error: code = NotFound desc = could not find container \"6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd\": container with ID starting with 6617e5cf99b5968ce61aa1739e8ccc770b6ef570d5ea3947ecb10e44f6bb70cd not found: ID does not exist"
Jul 24 17:42:46.618228 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618147    2575 scope.go:117] "RemoveContainer" containerID="4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"
Jul 24 17:42:46.618448 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618416    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4"} err="failed to get container status \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": rpc error: code = NotFound desc = could not find container \"4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4\": container with ID starting with 4770b55e2442f88262c180cbfc43e19982a3231e2875a61db9b269c96271c7f4 not found: ID does not exist"
Jul 24 17:42:46.618526 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618450    2575 scope.go:117] "RemoveContainer" containerID="31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"
Jul 24 17:42:46.618730 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618704    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938"} err="failed to get container status \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": rpc error: code = NotFound desc = could not find container \"31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938\": container with ID starting with 31abc95aa33860f826d7db28c59c23ef4a9097429e9235599b770656fef1d938 not found: ID does not exist"
Jul 24 17:42:46.618730 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618728    2575 scope.go:117] "RemoveContainer" containerID="c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"
Jul 24 17:42:46.618967 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618951    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52"} err="failed to get container status \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": rpc error: code = NotFound desc = could not find container \"c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52\": container with ID starting with c85d293b997d6bf1676884d827d45691457498c6d4419ef82e0aa730fa0ebf52 not found: ID does not exist"
Jul 24 17:42:46.618967 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.618968    2575 scope.go:117] "RemoveContainer" containerID="56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"
Jul 24 17:42:46.619186 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.619167    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b"} err="failed to get container status \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": rpc error: code = NotFound desc = could not find container \"56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b\": container with ID starting with 56fbc2fa87a65350ca8086a10b15a7f3ae06e16f182da7a52344598596bcde0b not found: ID does not exist"
Jul 24 17:42:46.696759 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696718    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696759 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696760    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696788    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696812    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696920    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696944    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-config\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696958    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.696994 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.696987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e6ae60c-a0da-4fce-9ece-94209736f20d-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697036    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697077    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697093    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4dr\" (UniqueName: \"kubernetes.io/projected/9e6ae60c-a0da-4fce-9ece-94209736f20d-kube-api-access-sg4dr\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697112    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697133    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697153    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697230 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697207    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697492 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697492 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697283    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e6ae60c-a0da-4fce-9ece-94209736f20d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.697492 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.697307    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.798761 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798664    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.798761 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798708    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.798761 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798740    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-config\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798766    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798791    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e6ae60c-a0da-4fce-9ece-94209736f20d-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798814    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798836    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798872    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798896    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4dr\" (UniqueName: \"kubernetes.io/projected/9e6ae60c-a0da-4fce-9ece-94209736f20d-kube-api-access-sg4dr\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798923    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798954    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.798987    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799029    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799066    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799101    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e6ae60c-a0da-4fce-9ece-94209736f20d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799163    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799195    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.799491 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799250    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.800302 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799779    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.800302 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.800107    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.800302 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.800142    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.802527 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.801030    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.802663 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.802529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.802663 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.802576    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.802956 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.799164    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.803231 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803039    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.803231 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803075    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9e6ae60c-a0da-4fce-9ece-94209736f20d-config-out\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.803398 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803234    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-config\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.803502 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803484    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9e6ae60c-a0da-4fce-9ece-94209736f20d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.803635 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803610    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.804037 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803965    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-web-config\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.804037 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.803987    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.804341 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.804320    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9e6ae60c-a0da-4fce-9ece-94209736f20d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.804484 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.804463    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.804900 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.804876    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.804900 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.804892    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.805394 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.805378    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9e6ae60c-a0da-4fce-9ece-94209736f20d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.807607 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.807578    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4dr\" (UniqueName: \"kubernetes.io/projected/9e6ae60c-a0da-4fce-9ece-94209736f20d-kube-api-access-sg4dr\") pod \"prometheus-k8s-0\" (UID: \"9e6ae60c-a0da-4fce-9ece-94209736f20d\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:46.912196 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:46.912155    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:47.060027 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:47.059919    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 24 17:42:47.063570 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:42:47.063545    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e6ae60c_a0da_4fce_9ece_94209736f20d.slice/crio-2304eb38388c72f8b9f11b0d54ac1b46a7bcce11a33415a702fcde9e39f80686 WatchSource:0}: Error finding container 2304eb38388c72f8b9f11b0d54ac1b46a7bcce11a33415a702fcde9e39f80686: Status 404 returned error can't find the container with id 2304eb38388c72f8b9f11b0d54ac1b46a7bcce11a33415a702fcde9e39f80686
Jul 24 17:42:47.546670 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:47.546632    2575 generic.go:356] "Generic (PLEG): container finished" podID="9e6ae60c-a0da-4fce-9ece-94209736f20d" containerID="50a64d41d442a169c9ac7c4fb45c8655aaf9f9197ba632aaeaf3fc036638fe42" exitCode=0
Jul 24 17:42:47.547131 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:47.546724    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerDied","Data":"50a64d41d442a169c9ac7c4fb45c8655aaf9f9197ba632aaeaf3fc036638fe42"}
Jul 24 17:42:47.547131 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:47.546797    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"2304eb38388c72f8b9f11b0d54ac1b46a7bcce11a33415a702fcde9e39f80686"}
Jul 24 17:42:48.142548 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.142463    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60e25139-0fd7-4a21-a4c3-0a796262e8b7" path="/var/lib/kubelet/pods/60e25139-0fd7-4a21-a4c3-0a796262e8b7/volumes"
Jul 24 17:42:48.557147 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.557100    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"8201e5523721e6be1d95dbd6fec3fc0f0183de671a64b941c208b2946bb639ba"}
Jul 24 17:42:48.557828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.557193    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"60032809f925360dceda64c18bc72874b57fe56b3641c48597081e868c04560f"}
Jul 24 17:42:48.557828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.557231    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"5f9480312917afae35feb73cdff799beab6c49b0d2278b020e59e36ed4a8f61f"}
Jul 24 17:42:48.557828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.557306    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"44c25940883a3aacd62fb148a2c1dacafa7ab4229d2ae7e6afa29099bc7567f9"}
Jul 24 17:42:48.557828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.557343    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"61c1e2ac654bbb55ac366ada7259c89bcbc61e9aa2ce988b909c6d0698c409ee"}
Jul 24 17:42:48.557828 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.557381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9e6ae60c-a0da-4fce-9ece-94209736f20d","Type":"ContainerStarted","Data":"84ef1b1b7b089fd6611f608014a8d37dca02cb51b4314459164da48f55c27a46"}
Jul 24 17:42:48.592990 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:48.592921    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.573305908 podStartE2EDuration="2.592897109s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.591201ms" isStatefulPod=false podCreationTimestamp="2026-07-24 17:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:42:48.591026034 +0000 UTC m=+366.994967290" watchObservedRunningTime="2026-07-24 17:42:48.592897109 +0000 UTC m=+366.996838339"
Jul 24 17:42:51.912342 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:51.912289    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:42:55.879837 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:55.879795    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:42:55.882142 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:55.882116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbeb29f5-efec-44dc-aed1-21dc7e02dca4-metrics-certs\") pod \"network-metrics-daemon-x2cmc\" (UID: \"cbeb29f5-efec-44dc-aed1-21dc7e02dca4\") " pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:42:56.135986 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:56.135909    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lvv7h\""
Jul 24 17:42:56.144318 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:56.144280    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-x2cmc"
Jul 24 17:42:56.284782 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:56.284743    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-x2cmc"]
Jul 24 17:42:56.288980 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:42:56.288953    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbeb29f5_efec_44dc_aed1_21dc7e02dca4.slice/crio-9210bf7b8c85c551e9e5e4ee2c23114bdf9717a0698a39f58a5ff8d9f826439c WatchSource:0}: Error finding container 9210bf7b8c85c551e9e5e4ee2c23114bdf9717a0698a39f58a5ff8d9f826439c: Status 404 returned error can't find the container with id 9210bf7b8c85c551e9e5e4ee2c23114bdf9717a0698a39f58a5ff8d9f826439c
Jul 24 17:42:56.290941 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:56.290924    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:42:56.591465 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:56.591428    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2cmc" event={"ID":"cbeb29f5-efec-44dc-aed1-21dc7e02dca4","Type":"ContainerStarted","Data":"9210bf7b8c85c551e9e5e4ee2c23114bdf9717a0698a39f58a5ff8d9f826439c"}
Jul 24 17:42:57.596778 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:57.596736    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2cmc" event={"ID":"cbeb29f5-efec-44dc-aed1-21dc7e02dca4","Type":"ContainerStarted","Data":"aaa7e46c66a914bf3dd84a96f5a2322b0db414364a07c6ae2ca85a844d6a7ef7"}
Jul 24 17:42:57.597186 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:57.596821    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-x2cmc" event={"ID":"cbeb29f5-efec-44dc-aed1-21dc7e02dca4","Type":"ContainerStarted","Data":"a8800ccc64369937985ad982177a59635b91f59403a50c636a6565aae49d155c"}
Jul 24 17:42:57.613152 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:42:57.613093    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-x2cmc" podStartSLOduration=373.7855807 podStartE2EDuration="6m14.613076035s" totalImagesPullingTime="827.495335ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:36:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:42:57.611227494 +0000 UTC m=+376.015168723" watchObservedRunningTime="2026-07-24 17:42:57.613076035 +0000 UTC m=+376.017017263"
Jul 24 17:43:46.912542 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:43:46.912499    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:43:46.928762 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:43:46.928732    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:43:47.813061 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:43:47.813034    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 24 17:46:09.291277 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.291181    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"]
Jul 24 17:46:09.294543 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.294527    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.301732 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.301714    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 17:46:09.302161 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.302145    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 17:46:09.302201 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.302166    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 17:46:09.302512 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.302495    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jth4d\""
Jul 24 17:46:09.304017 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.304003    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 17:46:09.336043 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.336006    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"]
Jul 24 17:46:09.336286 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.336248    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/468882ed-2cc3-458a-8fde-48d9fb4e8c38-tmp\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.336381 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.336296    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/468882ed-2cc3-458a-8fde-48d9fb4e8c38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.336381 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.336333    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/468882ed-2cc3-458a-8fde-48d9fb4e8c38-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.336381 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.336366    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/468882ed-2cc3-458a-8fde-48d9fb4e8c38-manifests\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.336535 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.336456    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz6kt\" (UniqueName: \"kubernetes.io/projected/468882ed-2cc3-458a-8fde-48d9fb4e8c38-kube-api-access-hz6kt\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.437608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.437574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/468882ed-2cc3-458a-8fde-48d9fb4e8c38-tmp\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.437608 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.437612    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/468882ed-2cc3-458a-8fde-48d9fb4e8c38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.437880 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.437639    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/468882ed-2cc3-458a-8fde-48d9fb4e8c38-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.437880 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.437673    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/468882ed-2cc3-458a-8fde-48d9fb4e8c38-manifests\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.437880 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.437731    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz6kt\" (UniqueName: \"kubernetes.io/projected/468882ed-2cc3-458a-8fde-48d9fb4e8c38-kube-api-access-hz6kt\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.438100 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.438078    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/468882ed-2cc3-458a-8fde-48d9fb4e8c38-manifests\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.439942 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.439914    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/468882ed-2cc3-458a-8fde-48d9fb4e8c38-tmp\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.440170 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.440152    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/468882ed-2cc3-458a-8fde-48d9fb4e8c38-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.440386 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.440368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/468882ed-2cc3-458a-8fde-48d9fb4e8c38-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.449701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.449668    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz6kt\" (UniqueName: \"kubernetes.io/projected/468882ed-2cc3-458a-8fde-48d9fb4e8c38-kube-api-access-hz6kt\") pod \"opendatahub-operator-controller-manager-6b794f588d-rcgmt\" (UID: \"468882ed-2cc3-458a-8fde-48d9fb4e8c38\") " pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.605151 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.605058    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:09.744363 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:09.744286    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"]
Jul 24 17:46:09.747574 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:46:09.747534    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468882ed_2cc3_458a_8fde_48d9fb4e8c38.slice/crio-cf96e7f8a46ebf70a41fef47d2f96d19e55e17425853a97e2a07655a1bb4c8d9 WatchSource:0}: Error finding container cf96e7f8a46ebf70a41fef47d2f96d19e55e17425853a97e2a07655a1bb4c8d9: Status 404 returned error can't find the container with id cf96e7f8a46ebf70a41fef47d2f96d19e55e17425853a97e2a07655a1bb4c8d9
Jul 24 17:46:10.369011 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:10.368968    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt" event={"ID":"468882ed-2cc3-458a-8fde-48d9fb4e8c38","Type":"ContainerStarted","Data":"cf96e7f8a46ebf70a41fef47d2f96d19e55e17425853a97e2a07655a1bb4c8d9"}
Jul 24 17:46:13.392165 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:13.392126    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt" event={"ID":"468882ed-2cc3-458a-8fde-48d9fb4e8c38","Type":"ContainerStarted","Data":"87be5e9f0b9db45b0611fb3ca16a5e96a75d996732e50c9c13df7c179faf8f44"}
Jul 24 17:46:14.398317 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:14.398254    2575 generic.go:356] "Generic (PLEG): container finished" podID="468882ed-2cc3-458a-8fde-48d9fb4e8c38" containerID="87be5e9f0b9db45b0611fb3ca16a5e96a75d996732e50c9c13df7c179faf8f44" exitCode=0
Jul 24 17:46:14.398705 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:14.398344    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt" event={"ID":"468882ed-2cc3-458a-8fde-48d9fb4e8c38","Type":"ContainerDied","Data":"87be5e9f0b9db45b0611fb3ca16a5e96a75d996732e50c9c13df7c179faf8f44"}
Jul 24 17:46:15.404578 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:15.404488    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt" event={"ID":"468882ed-2cc3-458a-8fde-48d9fb4e8c38","Type":"ContainerStarted","Data":"22144d3bd00a3f0656d2c269d9bd7112534b9fc0dd4bbcdb3c4bb89c3d86268c"}
Jul 24 17:46:15.404997 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:15.404644    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:15.426836 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:15.426771    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt" podStartSLOduration=1.7830970640000001 podStartE2EDuration="6.426748491s" totalImagesPullingTime="3.55532644s" totalInitContainerRuntime="1.088324987s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:15.425677374 +0000 UTC m=+573.829618631" watchObservedRunningTime="2026-07-24 17:46:15.426748491 +0000 UTC m=+573.830689723"
Jul 24 17:46:26.411796 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:26.411764    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b794f588d-rcgmt"
Jul 24 17:46:29.562589 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.562554    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"]
Jul 24 17:46:29.572145 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.572113    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.575152 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.575120    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 17:46:29.575347 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.575129    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 17:46:29.575347 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.575199    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-l2f9s\""
Jul 24 17:46:29.581044 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.581017    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"]
Jul 24 17:46:29.611260 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.611215    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjrb\" (UniqueName: \"kubernetes.io/projected/92b081b3-0862-4445-8f79-9adadfd063c6-kube-api-access-kgjrb\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.611470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.611290    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b081b3-0862-4445-8f79-9adadfd063c6-tls-certs\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.611470 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.611372    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92b081b3-0862-4445-8f79-9adadfd063c6-tmp\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.703760 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.703731    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"]
Jul 24 17:46:29.706078 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.706051    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.708517 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.708491    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vtzzk\""
Jul 24 17:46:29.708669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.708636    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 17:46:29.708669 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.708656    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 17:46:29.708769 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.708642    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 17:46:29.708969 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.708953    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 17:46:29.709022 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.708961    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 17:46:29.712727 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.712701    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjrb\" (UniqueName: \"kubernetes.io/projected/92b081b3-0862-4445-8f79-9adadfd063c6-kube-api-access-kgjrb\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.712879 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.712751    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b081b3-0862-4445-8f79-9adadfd063c6-tls-certs\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.712956 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.712917    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92b081b3-0862-4445-8f79-9adadfd063c6-tmp\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.715447 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.715416    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92b081b3-0862-4445-8f79-9adadfd063c6-tmp\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.715778 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.715758    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92b081b3-0862-4445-8f79-9adadfd063c6-tls-certs\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.719413 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.719381    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"]
Jul 24 17:46:29.730851 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.730820    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjrb\" (UniqueName: \"kubernetes.io/projected/92b081b3-0862-4445-8f79-9adadfd063c6-kube-api-access-kgjrb\") pod \"kube-auth-proxy-546fb58d4c-7fdxd\" (UID: \"92b081b3-0862-4445-8f79-9adadfd063c6\") " pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.813724 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.813609    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-manager-config\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.813905 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.813726    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-metrics-cert\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.813905 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.813756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrswg\" (UniqueName: \"kubernetes.io/projected/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-kube-api-access-xrswg\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.813905 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.813877    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-cert\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.883688 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.883651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"
Jul 24 17:46:29.914818 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.914779    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-manager-config\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.914818 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.914830    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-metrics-cert\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.915077 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.914885    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrswg\" (UniqueName: \"kubernetes.io/projected/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-kube-api-access-xrswg\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.915077 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.915030    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-cert\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.915568 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.915545    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-manager-config\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.917501 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.917474    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-metrics-cert\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.917660 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.917536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-cert\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:29.924619 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:29.924583    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrswg\" (UniqueName: \"kubernetes.io/projected/77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7-kube-api-access-xrswg\") pod \"lws-controller-manager-5b4bb5cddb-bcptj\" (UID: \"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7\") " pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:30.025998 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:30.025806    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd"]
Jul 24 17:46:30.028915 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:46:30.028881    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92b081b3_0862_4445_8f79_9adadfd063c6.slice/crio-a996fdb331d01a4a3f874c14b776afae1f98e53369613bf492a0a1d5f762498f WatchSource:0}: Error finding container a996fdb331d01a4a3f874c14b776afae1f98e53369613bf492a0a1d5f762498f: Status 404 returned error can't find the container with id a996fdb331d01a4a3f874c14b776afae1f98e53369613bf492a0a1d5f762498f
Jul 24 17:46:30.032106 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:30.032082    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:30.225720 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:30.225693    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"]
Jul 24 17:46:30.228337 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:46:30.228306    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ebb0f4_a143_49b1_8a5f_3ff538c5a6e7.slice/crio-8af1c3f175badcd180e8f8c60b7f57c4e18515cc2876d02721e8456038654b98 WatchSource:0}: Error finding container 8af1c3f175badcd180e8f8c60b7f57c4e18515cc2876d02721e8456038654b98: Status 404 returned error can't find the container with id 8af1c3f175badcd180e8f8c60b7f57c4e18515cc2876d02721e8456038654b98
Jul 24 17:46:30.474729 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:30.474684    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj" event={"ID":"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7","Type":"ContainerStarted","Data":"8af1c3f175badcd180e8f8c60b7f57c4e18515cc2876d02721e8456038654b98"}
Jul 24 17:46:30.475833 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:30.475807    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd" event={"ID":"92b081b3-0862-4445-8f79-9adadfd063c6","Type":"ContainerStarted","Data":"a996fdb331d01a4a3f874c14b776afae1f98e53369613bf492a0a1d5f762498f"}
Jul 24 17:46:32.487862 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:32.487815    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd" event={"ID":"92b081b3-0862-4445-8f79-9adadfd063c6","Type":"ContainerStarted","Data":"9558d34138f20c1207d73580441bd31f18d8ad3e29078be3961116921d30dcad"}
Jul 24 17:46:32.506988 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:32.506920    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-546fb58d4c-7fdxd" podStartSLOduration=1.201899534 podStartE2EDuration="3.506896137s" totalImagesPullingTime="2.304996603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:32.504195846 +0000 UTC m=+590.908137076" watchObservedRunningTime="2026-07-24 17:46:32.506896137 +0000 UTC m=+590.910837367"
Jul 24 17:46:34.499330 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:34.499289    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj" event={"ID":"77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7","Type":"ContainerStarted","Data":"1a238b56b7ebaf84f98d73d9ea92281cc50b2c2c441188d5bf3714819c2c645e"}
Jul 24 17:46:34.499786 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:34.499368    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:46:34.519884 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:34.519825    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj" podStartSLOduration=2.288354858 podStartE2EDuration="5.519807247s" totalImagesPullingTime="3.231452389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:46:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:46:34.518651865 +0000 UTC m=+592.922593102" watchObservedRunningTime="2026-07-24 17:46:34.519807247 +0000 UTC m=+592.923748476"
Jul 24 17:46:42.053990 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:42.053960    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:46:42.054523 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:42.054220    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:46:45.506927 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:46:45.506895    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b4bb5cddb-bcptj"
Jul 24 17:48:08.921258 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.921214    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"]
Jul 24 17:48:08.924370 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.924349    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:08.926909 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.926878    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 24 17:48:08.927068 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.927029    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 17:48:08.927402 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.927386    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 17:48:08.927832 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.927817    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2ht5m\""
Jul 24 17:48:08.939711 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:08.939679    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"]
Jul 24 17:48:09.004227 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.004193    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x82m\" (UniqueName: \"kubernetes.io/projected/25037198-0876-44b9-ac61-d04943afaf77-kube-api-access-7x82m\") pod \"dns-operator-controller-manager-864dd78c4c-nql2s\" (UID: \"25037198-0876-44b9-ac61-d04943afaf77\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:09.105277 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.105235    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x82m\" (UniqueName: \"kubernetes.io/projected/25037198-0876-44b9-ac61-d04943afaf77-kube-api-access-7x82m\") pod \"dns-operator-controller-manager-864dd78c4c-nql2s\" (UID: \"25037198-0876-44b9-ac61-d04943afaf77\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:09.119328 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.119286    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x82m\" (UniqueName: \"kubernetes.io/projected/25037198-0876-44b9-ac61-d04943afaf77-kube-api-access-7x82m\") pod \"dns-operator-controller-manager-864dd78c4c-nql2s\" (UID: \"25037198-0876-44b9-ac61-d04943afaf77\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:09.235474 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.235384    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:09.372544 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.372515    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"]
Jul 24 17:48:09.375249 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:48:09.375221    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25037198_0876_44b9_ac61_d04943afaf77.slice/crio-12b28b8ccf6846c95726d18ea393e92555bf33f974a267802b3a82360c8ec6e5 WatchSource:0}: Error finding container 12b28b8ccf6846c95726d18ea393e92555bf33f974a267802b3a82360c8ec6e5: Status 404 returned error can't find the container with id 12b28b8ccf6846c95726d18ea393e92555bf33f974a267802b3a82360c8ec6e5
Jul 24 17:48:09.377255 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.377238    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 17:48:09.935644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:09.935609    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s" event={"ID":"25037198-0876-44b9-ac61-d04943afaf77","Type":"ContainerStarted","Data":"12b28b8ccf6846c95726d18ea393e92555bf33f974a267802b3a82360c8ec6e5"}
Jul 24 17:48:11.947839 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:11.947798    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s" event={"ID":"25037198-0876-44b9-ac61-d04943afaf77","Type":"ContainerStarted","Data":"157af45db5ef66d3fa122095af8d1efaba8cc890bd0baf186e1103684ed40e1c"}
Jul 24 17:48:11.948284 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:11.947980    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:11.967396 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:11.967346    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s" podStartSLOduration=2.049331772 podStartE2EDuration="3.967329395s" totalImagesPullingTime="1.917997623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:11.965986236 +0000 UTC m=+690.369927466" watchObservedRunningTime="2026-07-24 17:48:11.967329395 +0000 UTC m=+690.371270817"
Jul 24 17:48:13.454705 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.454661    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"]
Jul 24 17:48:13.458348 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.458322    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.461183 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.461157    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sd8r8\""
Jul 24 17:48:13.461369 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.461166    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 24 17:48:13.461369 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.461168    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 24 17:48:13.467576 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.467549    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"]
Jul 24 17:48:13.546537 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.546502    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/167b8179-0965-444d-8055-f9cbdea5aa6d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.546537 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.546549    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/167b8179-0965-444d-8055-f9cbdea5aa6d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.546780 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.546586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjlvg\" (UniqueName: \"kubernetes.io/projected/167b8179-0965-444d-8055-f9cbdea5aa6d-kube-api-access-fjlvg\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.647124 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.647081    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/167b8179-0965-444d-8055-f9cbdea5aa6d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.647343 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.647132    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/167b8179-0965-444d-8055-f9cbdea5aa6d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.647343 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.647168    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjlvg\" (UniqueName: \"kubernetes.io/projected/167b8179-0965-444d-8055-f9cbdea5aa6d-kube-api-access-fjlvg\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.647904 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.647881    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/167b8179-0965-444d-8055-f9cbdea5aa6d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.649548 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.649529    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/167b8179-0965-444d-8055-f9cbdea5aa6d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.656639 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.656615    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjlvg\" (UniqueName: \"kubernetes.io/projected/167b8179-0965-444d-8055-f9cbdea5aa6d-kube-api-access-fjlvg\") pod \"kuadrant-console-plugin-66b5774985-x426q\" (UID: \"167b8179-0965-444d-8055-f9cbdea5aa6d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.775787 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.775684    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"
Jul 24 17:48:13.914318 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.914290    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x426q"]
Jul 24 17:48:13.916932 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:48:13.916902    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod167b8179_0965_444d_8055_f9cbdea5aa6d.slice/crio-320d31dca8c7179a4e59c43ca3c4642cc204cf539c60821a03c965d3a1aa7332 WatchSource:0}: Error finding container 320d31dca8c7179a4e59c43ca3c4642cc204cf539c60821a03c965d3a1aa7332: Status 404 returned error can't find the container with id 320d31dca8c7179a4e59c43ca3c4642cc204cf539c60821a03c965d3a1aa7332
Jul 24 17:48:13.958369 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:13.958334    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q" event={"ID":"167b8179-0965-444d-8055-f9cbdea5aa6d","Type":"ContainerStarted","Data":"320d31dca8c7179a4e59c43ca3c4642cc204cf539c60821a03c965d3a1aa7332"}
Jul 24 17:48:22.955114 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:22.955069    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-nql2s"
Jul 24 17:48:25.233395 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.233356    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"]
Jul 24 17:48:25.306461 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.306424    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"]
Jul 24 17:48:25.306631 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.306523    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:48:25.309277 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.309240    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-grk2p\""
Jul 24 17:48:25.462586 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.462548    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dbbn\" (UniqueName: \"kubernetes.io/projected/40d7aa2b-b0e0-4d2d-a30d-745d2abc849d-kube-api-access-4dbbn\") pod \"limitador-operator-controller-manager-859d7b78bd-k6kq5\" (UID: \"40d7aa2b-b0e0-4d2d-a30d-745d2abc849d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:48:25.563375 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.563285    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dbbn\" (UniqueName: \"kubernetes.io/projected/40d7aa2b-b0e0-4d2d-a30d-745d2abc849d-kube-api-access-4dbbn\") pod \"limitador-operator-controller-manager-859d7b78bd-k6kq5\" (UID: \"40d7aa2b-b0e0-4d2d-a30d-745d2abc849d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:48:25.584639 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.584608    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dbbn\" (UniqueName: \"kubernetes.io/projected/40d7aa2b-b0e0-4d2d-a30d-745d2abc849d-kube-api-access-4dbbn\") pod \"limitador-operator-controller-manager-859d7b78bd-k6kq5\" (UID: \"40d7aa2b-b0e0-4d2d-a30d-745d2abc849d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:48:25.619465 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:25.619420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:48:35.796034 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:35.795985    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"]
Jul 24 17:48:35.797741 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:48:35.797697    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d7aa2b_b0e0_4d2d_a30d_745d2abc849d.slice/crio-c856718c6f6b168d7e9de2d182b7d4b75b68f8bb5cc44b46445de40c7b140989 WatchSource:0}: Error finding container c856718c6f6b168d7e9de2d182b7d4b75b68f8bb5cc44b46445de40c7b140989: Status 404 returned error can't find the container with id c856718c6f6b168d7e9de2d182b7d4b75b68f8bb5cc44b46445de40c7b140989
Jul 24 17:48:36.084535 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:36.084497    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5" event={"ID":"40d7aa2b-b0e0-4d2d-a30d-745d2abc849d","Type":"ContainerStarted","Data":"c856718c6f6b168d7e9de2d182b7d4b75b68f8bb5cc44b46445de40c7b140989"}
Jul 24 17:48:36.086144 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:36.086111    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q" event={"ID":"167b8179-0965-444d-8055-f9cbdea5aa6d","Type":"ContainerStarted","Data":"e2669de876d20017fded2ff0f3d81198b3ba9b6e0c7f32ae79961e4ee1be3a9f"}
Jul 24 17:48:36.102370 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:36.102311    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x426q" podStartSLOduration=1.251368807 podStartE2EDuration="23.102294111s" totalImagesPullingTime="21.850925304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:36.10050675 +0000 UTC m=+714.504447972" watchObservedRunningTime="2026-07-24 17:48:36.102294111 +0000 UTC m=+714.506235340"
Jul 24 17:48:40.106874 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:40.106839    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5" event={"ID":"40d7aa2b-b0e0-4d2d-a30d-745d2abc849d","Type":"ContainerStarted","Data":"1328de7e87d8a8ceff7b835e0ae0230601e82b031d3cd826fa408fa0527b8731"}
Jul 24 17:48:40.107309 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:40.106991    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:48:40.124920 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:40.124861    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5" podStartSLOduration=11.352127076 podStartE2EDuration="15.124844938s" totalImagesPullingTime="3.772717862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:48:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:48:40.122720822 +0000 UTC m=+718.526662063" watchObservedRunningTime="2026-07-24 17:48:40.124844938 +0000 UTC m=+718.528786166"
Jul 24 17:48:51.113611 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:48:51.113580    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-k6kq5"
Jul 24 17:49:09.760529 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.760495    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:49:09.763057 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.763036    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:09.766425 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.766401    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 24 17:49:09.776587 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.776560    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:49:09.810127 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.810087    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:49:09.874758 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.874720    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f952578c-31b7-4e67-bb27-1d402585b995-config-file\") pod \"limitador-limitador-54d6d988d5-wmlt9\" (UID: \"f952578c-31b7-4e67-bb27-1d402585b995\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:09.874972 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.874819    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptn8b\" (UniqueName: \"kubernetes.io/projected/f952578c-31b7-4e67-bb27-1d402585b995-kube-api-access-ptn8b\") pod \"limitador-limitador-54d6d988d5-wmlt9\" (UID: \"f952578c-31b7-4e67-bb27-1d402585b995\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:09.976357 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.976315    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f952578c-31b7-4e67-bb27-1d402585b995-config-file\") pod \"limitador-limitador-54d6d988d5-wmlt9\" (UID: \"f952578c-31b7-4e67-bb27-1d402585b995\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:09.976551 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.976412    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptn8b\" (UniqueName: \"kubernetes.io/projected/f952578c-31b7-4e67-bb27-1d402585b995-kube-api-access-ptn8b\") pod \"limitador-limitador-54d6d988d5-wmlt9\" (UID: \"f952578c-31b7-4e67-bb27-1d402585b995\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:09.977055 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.977033    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f952578c-31b7-4e67-bb27-1d402585b995-config-file\") pod \"limitador-limitador-54d6d988d5-wmlt9\" (UID: \"f952578c-31b7-4e67-bb27-1d402585b995\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:09.985219 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:09.985185    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptn8b\" (UniqueName: \"kubernetes.io/projected/f952578c-31b7-4e67-bb27-1d402585b995-kube-api-access-ptn8b\") pod \"limitador-limitador-54d6d988d5-wmlt9\" (UID: \"f952578c-31b7-4e67-bb27-1d402585b995\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:10.073291 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:10.073162    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:10.216605 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:10.216576    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:49:10.218778 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:49:10.218749    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf952578c_31b7_4e67_bb27_1d402585b995.slice/crio-3a1a8ab000fc86e51f9770479d15756181e5d8618283b892945c10dd1288b9f4 WatchSource:0}: Error finding container 3a1a8ab000fc86e51f9770479d15756181e5d8618283b892945c10dd1288b9f4: Status 404 returned error can't find the container with id 3a1a8ab000fc86e51f9770479d15756181e5d8618283b892945c10dd1288b9f4
Jul 24 17:49:10.244802 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:10.244766    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9" event={"ID":"f952578c-31b7-4e67-bb27-1d402585b995","Type":"ContainerStarted","Data":"3a1a8ab000fc86e51f9770479d15756181e5d8618283b892945c10dd1288b9f4"}
Jul 24 17:49:13.261507 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:13.261467    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9" event={"ID":"f952578c-31b7-4e67-bb27-1d402585b995","Type":"ContainerStarted","Data":"9b86894bedcbf07cc7288ef32fdc430738c940cb23f90d4658e48c56aea9e2a1"}
Jul 24 17:49:13.261933 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:13.261581    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:49:13.293494 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:13.293424    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9" podStartSLOduration=1.8436226310000001 podStartE2EDuration="4.293403014s" totalImagesPullingTime="2.449780383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:49:13.29109355 +0000 UTC m=+751.695034778" watchObservedRunningTime="2026-07-24 17:49:13.293403014 +0000 UTC m=+751.697344245"
Jul 24 17:49:24.267605 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:49:24.267569    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"
Jul 24 17:50:25.337921 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:50:25.337882    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:50:49.453645 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:50:49.453607    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:50:55.852354 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:50:55.852316    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:51:10.056092 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:10.056043    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:51:15.362646 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:15.362611    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:51:22.071900 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:22.071860    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:51:28.762227 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.762192    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"]
Jul 24 17:51:28.765643 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.765627    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.768477 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.768454    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 17:51:28.768477 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.768453    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 17:51:28.769584 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.769562    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bs9cs\""
Jul 24 17:51:28.769683 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.769567    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 17:51:28.775481 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.775460    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"]
Jul 24 17:51:28.886965 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.886932    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.887135 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.887013    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.887135 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.887049    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv8gk\" (UniqueName: \"kubernetes.io/projected/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-kube-api-access-dv8gk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.887135 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.887085    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.887135 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.887099    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.887135 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.887118    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988060 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988026    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988091    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988118    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv8gk\" (UniqueName: \"kubernetes.io/projected/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-kube-api-access-dv8gk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988496 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988454    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988568 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988549    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.988619 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.988592    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.990448 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.990420    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.990644 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.990626    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:28.996137 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:28.996116    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv8gk\" (UniqueName: \"kubernetes.io/projected/e89eed6d-01c6-40f5-9f30-b746c1e8b19f-kube-api-access-dv8gk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-6t72m\" (UID: \"e89eed6d-01c6-40f5-9f30-b746c1e8b19f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:29.104171 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:29.104079    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:29.236765 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:29.236726    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"]
Jul 24 17:51:29.238298 ip-10-0-140-161 kubenswrapper[2575]: W0724 17:51:29.238258    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode89eed6d_01c6_40f5_9f30_b746c1e8b19f.slice/crio-550579a70395f9684b07cfdd790580bf58f04f997a563b279c88640cccec3b47 WatchSource:0}: Error finding container 550579a70395f9684b07cfdd790580bf58f04f997a563b279c88640cccec3b47: Status 404 returned error can't find the container with id 550579a70395f9684b07cfdd790580bf58f04f997a563b279c88640cccec3b47
Jul 24 17:51:29.903247 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:29.903207    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m" event={"ID":"e89eed6d-01c6-40f5-9f30-b746c1e8b19f","Type":"ContainerStarted","Data":"550579a70395f9684b07cfdd790580bf58f04f997a563b279c88640cccec3b47"}
Jul 24 17:51:30.910299 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:30.910250    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m" event={"ID":"e89eed6d-01c6-40f5-9f30-b746c1e8b19f","Type":"ContainerStarted","Data":"8085fd2cc83098f7d04227f194e22bb350fcda12e5956941626905af8bdc35aa"}
Jul 24 17:51:30.945186 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:30.945128    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m" podStartSLOduration=1.944648453 podStartE2EDuration="2.945112453s" totalImagesPullingTime="1.000464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 17:51:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 17:51:30.943952878 +0000 UTC m=+889.347894106" watchObservedRunningTime="2026-07-24 17:51:30.945112453 +0000 UTC m=+889.349053679"
Jul 24 17:51:32.230524 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:32.230492    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:51:39.104873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:39.104786    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:39.104873 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:39.104840    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:39.117197 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:39.117172    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:39.961764 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:39.961733    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-6t72m"
Jul 24 17:51:42.084061 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:42.084033    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:51:42.085160 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:51:42.085139    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:52:52.269240 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:52:52.269202    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:53:04.376503 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:53:04.376417    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:53:13.778287 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:53:13.778224    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:54:57.962701 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:54:57.962609    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:55:40.058478 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:55:40.058435    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:55:49.262633 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:55:49.262587    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:56:00.164036 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:00.164002    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:56:11.671610 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:11.671575    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:56:23.883643 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:23.883607    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:56:33.360005 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:33.359966    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:56:42.116260 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:42.116226    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:56:42.118582 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:42.118546    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 17:56:46.160371 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:46.160331    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:56:56.266929 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:56:56.266892    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:57:18.065529 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:57:18.065487    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:57:28.563622 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:57:28.563583    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:57:52.973222 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:57:52.973136    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:58:05.300304 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:58:05.300252    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:58:21.066695 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:58:21.066650    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:58:30.669676 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:58:30.669639    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:58:40.978081 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:58:40.978037    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 17:58:53.162703 ip-10-0-140-161 kubenswrapper[2575]: I0724 17:58:53.162662    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:00:48.162676 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:00:48.162632    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:00:59.656228 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:00:59.656187    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:01:09.061887 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:09.061841    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:01:20.820823 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:20.820783    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:01:30.173361 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:30.173319    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:01:41.160780 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:41.160734    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:01:42.145475 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:42.145450    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:01:42.150855 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:42.150825    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:01:50.797569 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:01:50.797530    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:02:00.969695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:02:00.969661    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:02:13.060226 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:02:13.060186    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:02:21.772277 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:02:21.772235    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:02:33.157344 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:02:33.157307    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:02:53.972492 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:02:53.972454    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:03:03.175722 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.175678    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"]
Jul 24 18:03:03.179206 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.179180    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.181976 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.181951    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-803cded9-kserve-self-signed-certs\""
Jul 24 18:03:03.200852 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.200820    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"]
Jul 24 18:03:03.245294 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.245223    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw45s\" (UniqueName: \"kubernetes.io/projected/649cbed0-7229-4827-a18a-f49ce2d0ec96-kube-api-access-hw45s\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.245480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.245346    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cbed0-7229-4827-a18a-f49ce2d0ec96-tls-certs\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.245480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.245410    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-tmp-dir\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.245480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.245445    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-dshm\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.245589 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.245486    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-home\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.245589 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.245503    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-model-cache\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346406 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346373    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-tmp-dir\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346609 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346420    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-dshm\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346609 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346451    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-home\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346609 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346466    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-model-cache\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346609 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346514    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw45s\" (UniqueName: \"kubernetes.io/projected/649cbed0-7229-4827-a18a-f49ce2d0ec96-kube-api-access-hw45s\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346609 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346544    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cbed0-7229-4827-a18a-f49ce2d0ec96-tls-certs\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346873 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346839    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-tmp-dir\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.346932 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346890    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-home\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.347002 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.346985    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-model-cache\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.348889 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.348862    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-dshm\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.349090 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.349072    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cbed0-7229-4827-a18a-f49ce2d0ec96-tls-certs\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.355395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.355365    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw45s\" (UniqueName: \"kubernetes.io/projected/649cbed0-7229-4827-a18a-f49ce2d0ec96-kube-api-access-hw45s\") pod \"e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") " pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.488803 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.488704    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:03.626770 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:03.626712    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"]
Jul 24 18:03:03.631059 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:03:03.631026    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod649cbed0_7229_4827_a18a_f49ce2d0ec96.slice/crio-e380505a2af120be3aa0176fa1e6d5f1d58a63092c1ba5929ce87b38ff2bd145 WatchSource:0}: Error finding container e380505a2af120be3aa0176fa1e6d5f1d58a63092c1ba5929ce87b38ff2bd145: Status 404 returned error can't find the container with id e380505a2af120be3aa0176fa1e6d5f1d58a63092c1ba5929ce87b38ff2bd145
Jul 24 18:03:04.056450 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:04.056412    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:03:04.198865 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:04.198831    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh" event={"ID":"649cbed0-7229-4827-a18a-f49ce2d0ec96","Type":"ContainerStarted","Data":"5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437"}
Jul 24 18:03:04.198865 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:04.198870    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh" event={"ID":"649cbed0-7229-4827-a18a-f49ce2d0ec96","Type":"ContainerStarted","Data":"e380505a2af120be3aa0176fa1e6d5f1d58a63092c1ba5929ce87b38ff2bd145"}
Jul 24 18:03:04.219595 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:04.219525    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh" podStartSLOduration=1.219505515 podStartE2EDuration="1.219505515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:03:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:03:04.217698593 +0000 UTC m=+1582.621639824" watchObservedRunningTime="2026-07-24 18:03:04.219505515 +0000 UTC m=+1582.623446743"
Jul 24 18:03:13.489802 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:13.489753    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:13.490307 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:13.489823    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:13.502603 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:13.502572    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:14.259540 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:14.259503    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:03:16.353415 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:16.353371    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:03:18.839284 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.839218    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"]
Jul 24 18:03:18.842946 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.842918    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:18.845345 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.845318    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-803cded9-kserve-self-signed-certs\""
Jul 24 18:03:18.855555 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.855525    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"]
Jul 24 18:03:18.997891 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.997849    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-dshm\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:18.998079 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.997899    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-home\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:18.998079 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.997919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-model-cache\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:18.998079 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.998049    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0981e04-7f21-4065-9dd4-175a16dc9c80-tls-certs\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:18.998183 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.998088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/b0981e04-7f21-4065-9dd4-175a16dc9c80-kube-api-access-zxdbf\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:18.998183 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:18.998112    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-tmp-dir\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.098695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.098607    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0981e04-7f21-4065-9dd4-175a16dc9c80-tls-certs\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.098695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.098651    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/b0981e04-7f21-4065-9dd4-175a16dc9c80-kube-api-access-zxdbf\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.098695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.098672    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-tmp-dir\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.098695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.098697    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-dshm\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.098964 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.098738    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-home\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.098964 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.098757    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-model-cache\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.099112 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.099089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-tmp-dir\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.099173 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.099146    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-model-cache\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.099223 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.099175    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-home\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.101010 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.100991    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-dshm\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.101418 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.101398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0981e04-7f21-4065-9dd4-175a16dc9c80-tls-certs\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.106512 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.106487    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/b0981e04-7f21-4065-9dd4-175a16dc9c80-kube-api-access-zxdbf\") pod \"e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") " pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.153693 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.153644    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:19.297492 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:19.297362    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"]
Jul 24 18:03:19.299735 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:03:19.299706    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0981e04_7f21_4065_9dd4_175a16dc9c80.slice/crio-9d1cde2a1333171bf563bf10d88f08077f7f980247b5e856d592a04246c75331 WatchSource:0}: Error finding container 9d1cde2a1333171bf563bf10d88f08077f7f980247b5e856d592a04246c75331: Status 404 returned error can't find the container with id 9d1cde2a1333171bf563bf10d88f08077f7f980247b5e856d592a04246c75331
Jul 24 18:03:20.278859 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:20.278823    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms" event={"ID":"b0981e04-7f21-4065-9dd4-175a16dc9c80","Type":"ContainerStarted","Data":"abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1"}
Jul 24 18:03:20.279292 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:20.278868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms" event={"ID":"b0981e04-7f21-4065-9dd4-175a16dc9c80","Type":"ContainerStarted","Data":"9d1cde2a1333171bf563bf10d88f08077f7f980247b5e856d592a04246c75331"}
Jul 24 18:03:20.299059 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:20.299002    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms" podStartSLOduration=2.298987465 podStartE2EDuration="2.298987465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:03:20.297116654 +0000 UTC m=+1598.701057883" watchObservedRunningTime="2026-07-24 18:03:20.298987465 +0000 UTC m=+1598.702928693"
Jul 24 18:03:29.154134 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:29.154091    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:29.154661 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:29.154155    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:29.167097 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:29.167056    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:29.338775 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:29.338741    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:03:29.861209 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:29.861169    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:03:45.459917 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:45.459830    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"]
Jul 24 18:03:46.553651 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:46.553615    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"]
Jul 24 18:03:49.562879 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:03:49.562839    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:04:00.479570 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.479529    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh" podUID="649cbed0-7229-4827-a18a-f49ce2d0ec96" containerName="main" containerID="cri-o://5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437" gracePeriod=45
Jul 24 18:04:00.732208 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.732140    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:04:00.894198 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894159    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-model-cache\") pod \"649cbed0-7229-4827-a18a-f49ce2d0ec96\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") "
Jul 24 18:04:00.894437 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894223    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-tmp-dir\") pod \"649cbed0-7229-4827-a18a-f49ce2d0ec96\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") "
Jul 24 18:04:00.894437 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894247    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cbed0-7229-4827-a18a-f49ce2d0ec96-tls-certs\") pod \"649cbed0-7229-4827-a18a-f49ce2d0ec96\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") "
Jul 24 18:04:00.894437 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894294    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-dshm\") pod \"649cbed0-7229-4827-a18a-f49ce2d0ec96\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") "
Jul 24 18:04:00.894437 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894389    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw45s\" (UniqueName: \"kubernetes.io/projected/649cbed0-7229-4827-a18a-f49ce2d0ec96-kube-api-access-hw45s\") pod \"649cbed0-7229-4827-a18a-f49ce2d0ec96\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") "
Jul 24 18:04:00.894437 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894417    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-home\") pod \"649cbed0-7229-4827-a18a-f49ce2d0ec96\" (UID: \"649cbed0-7229-4827-a18a-f49ce2d0ec96\") "
Jul 24 18:04:00.894686 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894539    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-model-cache" (OuterVolumeSpecName: "model-cache") pod "649cbed0-7229-4827-a18a-f49ce2d0ec96" (UID: "649cbed0-7229-4827-a18a-f49ce2d0ec96"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:00.894686 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894557    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "649cbed0-7229-4827-a18a-f49ce2d0ec96" (UID: "649cbed0-7229-4827-a18a-f49ce2d0ec96"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:00.894794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894707    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:00.894794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894735    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:00.894794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.894730    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-home" (OuterVolumeSpecName: "home") pod "649cbed0-7229-4827-a18a-f49ce2d0ec96" (UID: "649cbed0-7229-4827-a18a-f49ce2d0ec96"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:00.896647 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.896614    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-dshm" (OuterVolumeSpecName: "dshm") pod "649cbed0-7229-4827-a18a-f49ce2d0ec96" (UID: "649cbed0-7229-4827-a18a-f49ce2d0ec96"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:00.896647 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.896638    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/649cbed0-7229-4827-a18a-f49ce2d0ec96-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "649cbed0-7229-4827-a18a-f49ce2d0ec96" (UID: "649cbed0-7229-4827-a18a-f49ce2d0ec96"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:04:00.896799 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.896609    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/649cbed0-7229-4827-a18a-f49ce2d0ec96-kube-api-access-hw45s" (OuterVolumeSpecName: "kube-api-access-hw45s") pod "649cbed0-7229-4827-a18a-f49ce2d0ec96" (UID: "649cbed0-7229-4827-a18a-f49ce2d0ec96"). InnerVolumeSpecName "kube-api-access-hw45s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:04:00.995888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.995790    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw45s\" (UniqueName: \"kubernetes.io/projected/649cbed0-7229-4827-a18a-f49ce2d0ec96-kube-api-access-hw45s\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:00.995888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.995826    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:00.995888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.995837    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/649cbed0-7229-4827-a18a-f49ce2d0ec96-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:00.995888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:00.995847    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/649cbed0-7229-4827-a18a-f49ce2d0ec96-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:01.493358 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.493325    2575 generic.go:356] "Generic (PLEG): container finished" podID="649cbed0-7229-4827-a18a-f49ce2d0ec96" containerID="5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437" exitCode=0
Jul 24 18:04:01.493855 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.493377    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"
Jul 24 18:04:01.493855 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.493381    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh" event={"ID":"649cbed0-7229-4827-a18a-f49ce2d0ec96","Type":"ContainerDied","Data":"5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437"}
Jul 24 18:04:01.493855 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.493535    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh" event={"ID":"649cbed0-7229-4827-a18a-f49ce2d0ec96","Type":"ContainerDied","Data":"e380505a2af120be3aa0176fa1e6d5f1d58a63092c1ba5929ce87b38ff2bd145"}
Jul 24 18:04:01.493855 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.493571    2575 scope.go:117] "RemoveContainer" containerID="5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437"
Jul 24 18:04:01.503378 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.503356    2575 scope.go:117] "RemoveContainer" containerID="5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437"
Jul 24 18:04:01.503730 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:04:01.503710    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437\": container with ID starting with 5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437 not found: ID does not exist" containerID="5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437"
Jul 24 18:04:01.503800 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.503739    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437"} err="failed to get container status \"5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437\": rpc error: code = NotFound desc = could not find container \"5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437\": container with ID starting with 5da12da3857b43ed9f81a4dc1268ffd619707ffcee98f71ea616e3eafb291437 not found: ID does not exist"
Jul 24 18:04:01.514582 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.514544    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"]
Jul 24 18:04:01.518455 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.518424    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-803cded9-kserve-5b4cb8fbd7-g8qvh"]
Jul 24 18:04:01.573792 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.573747    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms" podUID="b0981e04-7f21-4065-9dd4-175a16dc9c80" containerName="main" containerID="cri-o://abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1" gracePeriod=45
Jul 24 18:04:01.834009 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:01.833978    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:04:02.004872 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.004772    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-dshm\") pod \"b0981e04-7f21-4065-9dd4-175a16dc9c80\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") "
Jul 24 18:04:02.004872 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.004813    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0981e04-7f21-4065-9dd4-175a16dc9c80-tls-certs\") pod \"b0981e04-7f21-4065-9dd4-175a16dc9c80\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") "
Jul 24 18:04:02.005105 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.004892    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-home\") pod \"b0981e04-7f21-4065-9dd4-175a16dc9c80\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") "
Jul 24 18:04:02.005105 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.004923    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-model-cache\") pod \"b0981e04-7f21-4065-9dd4-175a16dc9c80\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") "
Jul 24 18:04:02.005105 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.004952    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-tmp-dir\") pod \"b0981e04-7f21-4065-9dd4-175a16dc9c80\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") "
Jul 24 18:04:02.005105 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.004987    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/b0981e04-7f21-4065-9dd4-175a16dc9c80-kube-api-access-zxdbf\") pod \"b0981e04-7f21-4065-9dd4-175a16dc9c80\" (UID: \"b0981e04-7f21-4065-9dd4-175a16dc9c80\") "
Jul 24 18:04:02.005313 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.005192    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-home" (OuterVolumeSpecName: "home") pod "b0981e04-7f21-4065-9dd4-175a16dc9c80" (UID: "b0981e04-7f21-4065-9dd4-175a16dc9c80"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:02.005313 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.005254    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0981e04-7f21-4065-9dd4-175a16dc9c80" (UID: "b0981e04-7f21-4065-9dd4-175a16dc9c80"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:02.005313 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.005247    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0981e04-7f21-4065-9dd4-175a16dc9c80" (UID: "b0981e04-7f21-4065-9dd4-175a16dc9c80"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:02.005436 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.005383    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:02.007085 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.007061    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-dshm" (OuterVolumeSpecName: "dshm") pod "b0981e04-7f21-4065-9dd4-175a16dc9c80" (UID: "b0981e04-7f21-4065-9dd4-175a16dc9c80"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:04:02.007196 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.007095    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0981e04-7f21-4065-9dd4-175a16dc9c80-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0981e04-7f21-4065-9dd4-175a16dc9c80" (UID: "b0981e04-7f21-4065-9dd4-175a16dc9c80"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:04:02.007196 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.007124    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0981e04-7f21-4065-9dd4-175a16dc9c80-kube-api-access-zxdbf" (OuterVolumeSpecName: "kube-api-access-zxdbf") pod "b0981e04-7f21-4065-9dd4-175a16dc9c80" (UID: "b0981e04-7f21-4065-9dd4-175a16dc9c80"). InnerVolumeSpecName "kube-api-access-zxdbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:04:02.105854 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.105819    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:02.105854 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.105849    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:02.105854 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.105861    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxdbf\" (UniqueName: \"kubernetes.io/projected/b0981e04-7f21-4065-9dd4-175a16dc9c80-kube-api-access-zxdbf\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:02.106101 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.105874    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0981e04-7f21-4065-9dd4-175a16dc9c80-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:02.106101 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.105883    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0981e04-7f21-4065-9dd4-175a16dc9c80-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:04:02.138446 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.138412    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="649cbed0-7229-4827-a18a-f49ce2d0ec96" path="/var/lib/kubelet/pods/649cbed0-7229-4827-a18a-f49ce2d0ec96/volumes"
Jul 24 18:04:02.500971 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.500933    2575 generic.go:356] "Generic (PLEG): container finished" podID="b0981e04-7f21-4065-9dd4-175a16dc9c80" containerID="abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1" exitCode=0
Jul 24 18:04:02.501449 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.500977    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"
Jul 24 18:04:02.501449 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.501014    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms" event={"ID":"b0981e04-7f21-4065-9dd4-175a16dc9c80","Type":"ContainerDied","Data":"abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1"}
Jul 24 18:04:02.501449 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.501054    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms" event={"ID":"b0981e04-7f21-4065-9dd4-175a16dc9c80","Type":"ContainerDied","Data":"9d1cde2a1333171bf563bf10d88f08077f7f980247b5e856d592a04246c75331"}
Jul 24 18:04:02.501449 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.501073    2575 scope.go:117] "RemoveContainer" containerID="abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1"
Jul 24 18:04:02.510549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.510530    2575 scope.go:117] "RemoveContainer" containerID="abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1"
Jul 24 18:04:02.510885 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:04:02.510864    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1\": container with ID starting with abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1 not found: ID does not exist" containerID="abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1"
Jul 24 18:04:02.510935 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.510895    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1"} err="failed to get container status \"abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1\": rpc error: code = NotFound desc = could not find container \"abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1\": container with ID starting with abdf49317ae268b4ad82febe439ba83acd4e7f9519510873f178c08ba83ef2f1 not found: ID does not exist"
Jul 24 18:04:02.520809 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.520775    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"]
Jul 24 18:04:02.525127 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:02.525097    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-803cded9-kserve-f8f496d99-qgxms"]
Jul 24 18:04:04.137227 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:04.137177    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0981e04-7f21-4065-9dd4-175a16dc9c80" path="/var/lib/kubelet/pods/b0981e04-7f21-4065-9dd4-175a16dc9c80/volumes"
Jul 24 18:04:04.358234 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:04.358194    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:04:10.760794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:10.760751    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:04:26.954989 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:26.954949    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:04:36.659756 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:36.659719    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:04:46.156093 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:46.156055    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:04:54.163295 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:04:54.163227    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:05:04.362798 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:05:04.362690    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:05:14.158397 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:05:14.158361    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:05:27.360907 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:05:27.360867    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:05:36.558477 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:05:36.558431    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:05:46.764295 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:05:46.764234    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:05:59.654713 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:05:59.654670    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:06:10.758205 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:06:10.758158    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:06:19.771199 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:06:19.771151    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:06:30.974745 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:06:30.974704    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:06:42.174997 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:06:42.174964    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:06:42.180898 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:06:42.180869    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:06:46.767835 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:06:46.767790    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:07:00.082191 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:07:00.082144    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:07:11.967958 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:07:11.967916    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:07:22.161956 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:07:22.161913    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:07:32.073798 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:07:32.073757    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:07:41.655185 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:07:41.655144    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:08:17.363064 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:08:17.363017    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:08:28.760008 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:08:28.759970    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:09:01.556217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:09:01.556178    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:09:12.599803 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:09:12.599758    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:09:45.452091 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:09:45.452054    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:09:56.870190 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:09:56.870150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:10:29.460458 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:10:29.460421    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:11:42.203688 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:11:42.203653    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:11:42.210785 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:11:42.210755    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:11:57.262456 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:11:57.262410    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:12:07.362900 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:12:07.362858    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:12:21.880031 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:12:21.879989    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:12:41.665690 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:12:41.665607    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:13:17.658568 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:17.658525    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:13:27.567715 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.567672    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"]
Jul 24 18:13:27.568118 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.568038    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="649cbed0-7229-4827-a18a-f49ce2d0ec96" containerName="main"
Jul 24 18:13:27.568118 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.568049    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="649cbed0-7229-4827-a18a-f49ce2d0ec96" containerName="main"
Jul 24 18:13:27.568118 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.568069    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0981e04-7f21-4065-9dd4-175a16dc9c80" containerName="main"
Jul 24 18:13:27.568118 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.568074    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0981e04-7f21-4065-9dd4-175a16dc9c80" containerName="main"
Jul 24 18:13:27.568307 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.568141    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="649cbed0-7229-4827-a18a-f49ce2d0ec96" containerName="main"
Jul 24 18:13:27.568307 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.568152    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0981e04-7f21-4065-9dd4-175a16dc9c80" containerName="main"
Jul 24 18:13:27.571314 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.571290    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.573784 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.573757    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a046f5ba\"/\"default-dockercfg-h45j2\""
Jul 24 18:13:27.574195 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.574175    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a046f5ba\"/\"openshift-service-ca.crt\""
Jul 24 18:13:27.574694 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.574668    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-a046f5ba\"/\"e2e-lifecycle-model-a046f5ba-kserve-self-signed-certs\""
Jul 24 18:13:27.574765 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.574724    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-a046f5ba\"/\"kube-root-ca.crt\""
Jul 24 18:13:27.581125 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.581089    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"]
Jul 24 18:13:27.638046 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.638007    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-home\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.638234 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.638074    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tmp-dir\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.638234 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.638121    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-dshm\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.638234 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.638139    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tls-certs\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.638234 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.638181    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj2w4\" (UniqueName: \"kubernetes.io/projected/296ead4c-4ff4-4fa0-920f-9752df2e7b61-kube-api-access-lj2w4\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.638397 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.638250    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-model-cache\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.739661 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.739613    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-model-cache\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.739888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.739684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-home\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.739888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.739746    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tmp-dir\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.739888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.739805    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-dshm\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.739888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.739828    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tls-certs\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.739888 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.739854    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj2w4\" (UniqueName: \"kubernetes.io/projected/296ead4c-4ff4-4fa0-920f-9752df2e7b61-kube-api-access-lj2w4\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.740138 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.740089    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-model-cache\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.740193 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.740149    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-home\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.740193 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.740172    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tmp-dir\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.742087 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.742064    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-dshm\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.742403 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.742383    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tls-certs\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.747951 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.747925    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj2w4\" (UniqueName: \"kubernetes.io/projected/296ead4c-4ff4-4fa0-920f-9752df2e7b61-kube-api-access-lj2w4\") pod \"e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") " pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:27.882871 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:27.882760    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:28.029219 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:28.029181    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"]
Jul 24 18:13:28.033007 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:13:28.032969    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod296ead4c_4ff4_4fa0_920f_9752df2e7b61.slice/crio-b6be24b9e0bfea0378a598227d87177ff8cac2fb7174fc63bac1671e7924b93d WatchSource:0}: Error finding container b6be24b9e0bfea0378a598227d87177ff8cac2fb7174fc63bac1671e7924b93d: Status 404 returned error can't find the container with id b6be24b9e0bfea0378a598227d87177ff8cac2fb7174fc63bac1671e7924b93d
Jul 24 18:13:28.213180 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:28.213136    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4" event={"ID":"296ead4c-4ff4-4fa0-920f-9752df2e7b61","Type":"ContainerStarted","Data":"70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170"}
Jul 24 18:13:28.213180 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:28.213189    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4" event={"ID":"296ead4c-4ff4-4fa0-920f-9752df2e7b61","Type":"ContainerStarted","Data":"b6be24b9e0bfea0378a598227d87177ff8cac2fb7174fc63bac1671e7924b93d"}
Jul 24 18:13:28.237341 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:28.237286    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4" podStartSLOduration=1.237252737 podStartE2EDuration="1.237252737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:13:28.236448634 +0000 UTC m=+2206.640389863" watchObservedRunningTime="2026-07-24 18:13:28.237252737 +0000 UTC m=+2206.641193943"
Jul 24 18:13:30.872668 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:30.872622    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:13:37.883260 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:37.883203    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:37.883716 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:37.883298    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:37.896357 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:37.896323    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:38.271026 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:38.270996    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:13:47.065039 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:13:47.064996    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:14:05.657659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:05.657568    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:14:14.273995 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:14.273956    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:14:20.181005 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:20.180965    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"]
Jul 24 18:14:35.201365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.201318    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4" podUID="296ead4c-4ff4-4fa0-920f-9752df2e7b61" containerName="main" containerID="cri-o://70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170" gracePeriod=45
Jul 24 18:14:35.458710 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.458627    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:14:35.547002 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.546969    2575 generic.go:356] "Generic (PLEG): container finished" podID="296ead4c-4ff4-4fa0-920f-9752df2e7b61" containerID="70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170" exitCode=0
Jul 24 18:14:35.547191 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.547015    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"
Jul 24 18:14:35.547191 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.547050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4" event={"ID":"296ead4c-4ff4-4fa0-920f-9752df2e7b61","Type":"ContainerDied","Data":"70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170"}
Jul 24 18:14:35.547191 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.547086    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4" event={"ID":"296ead4c-4ff4-4fa0-920f-9752df2e7b61","Type":"ContainerDied","Data":"b6be24b9e0bfea0378a598227d87177ff8cac2fb7174fc63bac1671e7924b93d"}
Jul 24 18:14:35.547191 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.547105    2575 scope.go:117] "RemoveContainer" containerID="70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170"
Jul 24 18:14:35.558256 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.558219    2575 scope.go:117] "RemoveContainer" containerID="70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170"
Jul 24 18:14:35.558833 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:14:35.558810    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170\": container with ID starting with 70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170 not found: ID does not exist" containerID="70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170"
Jul 24 18:14:35.558919 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.558849    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170"} err="failed to get container status \"70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170\": rpc error: code = NotFound desc = could not find container \"70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170\": container with ID starting with 70d77e8f9963b0e800a0e4d5b8754b729b4b8a218389c99b97d6f8b75930e170 not found: ID does not exist"
Jul 24 18:14:35.569229 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569187    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tmp-dir\") pod \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") "
Jul 24 18:14:35.569393 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569251    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-home\") pod \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") "
Jul 24 18:14:35.569393 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569287    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-dshm\") pod \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") "
Jul 24 18:14:35.569393 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569316    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-model-cache\") pod \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") "
Jul 24 18:14:35.569393 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569368    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tls-certs\") pod \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") "
Jul 24 18:14:35.569604 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569414    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lj2w4\" (UniqueName: \"kubernetes.io/projected/296ead4c-4ff4-4fa0-920f-9752df2e7b61-kube-api-access-lj2w4\") pod \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\" (UID: \"296ead4c-4ff4-4fa0-920f-9752df2e7b61\") "
Jul 24 18:14:35.569604 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569544    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "296ead4c-4ff4-4fa0-920f-9752df2e7b61" (UID: "296ead4c-4ff4-4fa0-920f-9752df2e7b61"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:14:35.569604 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569586    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-model-cache" (OuterVolumeSpecName: "model-cache") pod "296ead4c-4ff4-4fa0-920f-9752df2e7b61" (UID: "296ead4c-4ff4-4fa0-920f-9752df2e7b61"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:14:35.569746 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569525    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-home" (OuterVolumeSpecName: "home") pod "296ead4c-4ff4-4fa0-920f-9752df2e7b61" (UID: "296ead4c-4ff4-4fa0-920f-9752df2e7b61"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:14:35.569817 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569790    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:14:35.569817 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569816    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:14:35.569974 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.569830    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:14:35.571722 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.571678    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-dshm" (OuterVolumeSpecName: "dshm") pod "296ead4c-4ff4-4fa0-920f-9752df2e7b61" (UID: "296ead4c-4ff4-4fa0-920f-9752df2e7b61"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:14:35.571851 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.571756    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "296ead4c-4ff4-4fa0-920f-9752df2e7b61" (UID: "296ead4c-4ff4-4fa0-920f-9752df2e7b61"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:14:35.571851 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.571772    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296ead4c-4ff4-4fa0-920f-9752df2e7b61-kube-api-access-lj2w4" (OuterVolumeSpecName: "kube-api-access-lj2w4") pod "296ead4c-4ff4-4fa0-920f-9752df2e7b61" (UID: "296ead4c-4ff4-4fa0-920f-9752df2e7b61"). InnerVolumeSpecName "kube-api-access-lj2w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:14:35.671412 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.671369    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296ead4c-4ff4-4fa0-920f-9752df2e7b61-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:14:35.671412 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.671402    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296ead4c-4ff4-4fa0-920f-9752df2e7b61-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:14:35.671412 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.671421    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lj2w4\" (UniqueName: \"kubernetes.io/projected/296ead4c-4ff4-4fa0-920f-9752df2e7b61-kube-api-access-lj2w4\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:14:35.869725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.869682    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"]
Jul 24 18:14:35.874377 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:35.874343    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-a046f5ba/e2e-lifecycle-model-a046f5ba-kserve-6f7cccf64c-lh6q4"]
Jul 24 18:14:36.137708 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:14:36.137625    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="296ead4c-4ff4-4fa0-920f-9752df2e7b61" path="/var/lib/kubelet/pods/296ead4c-4ff4-4fa0-920f-9752df2e7b61/volumes"
Jul 24 18:15:07.363254 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:15:07.363213    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:15:19.878052 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:15:19.878008    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:16:10.070536 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:10.070446    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:16:20.448860 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.448822    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"]
Jul 24 18:16:20.449348 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.449213    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296ead4c-4ff4-4fa0-920f-9752df2e7b61" containerName="main"
Jul 24 18:16:20.449348 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.449226    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="296ead4c-4ff4-4fa0-920f-9752df2e7b61" containerName="main"
Jul 24 18:16:20.449348 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.449313    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="296ead4c-4ff4-4fa0-920f-9752df2e7b61" containerName="main"
Jul 24 18:16:20.452477 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.452460    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.455542 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.455512    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b3ffbc\"/\"openshift-service-ca.crt\""
Jul 24 18:16:20.455707 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.455574    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b3ffbc\"/\"test-model-b3ffbc-kserve-self-signed-certs\""
Jul 24 18:16:20.456319 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.456292    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b3ffbc\"/\"kube-root-ca.crt\""
Jul 24 18:16:20.456477 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.456334    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b3ffbc\"/\"default-dockercfg-ns758\""
Jul 24 18:16:20.464233 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.464205    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"]
Jul 24 18:16:20.533278 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.533241    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-model-cache\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.533456 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.533360    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-home\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.533456 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.533415    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjpj\" (UniqueName: \"kubernetes.io/projected/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-kube-api-access-2vjpj\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.533456 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.533436    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tmp-dir\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.533574 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.533508    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tls-certs\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.533574 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.533563    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-dshm\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.634522 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.634478    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-home\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.634767 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.634552    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjpj\" (UniqueName: \"kubernetes.io/projected/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-kube-api-access-2vjpj\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.634767 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.634574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tmp-dir\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.634767 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.634617    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tls-certs\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.634767 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.634660    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-dshm\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.634767 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.634711    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-model-cache\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.635043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.635005    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-home\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.635104 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.635037    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tmp-dir\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.635157 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.635112    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-model-cache\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.637069 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.637042    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-dshm\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.637336 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.637304    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tls-certs\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.643676 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.643648    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjpj\" (UniqueName: \"kubernetes.io/projected/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-kube-api-access-2vjpj\") pod \"test-model-b3ffbc-kserve-cff9fdbf-c5cqq\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") " pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.762806 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.762693    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:20.934059 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:20.934022    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"]
Jul 24 18:16:20.937341 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:16:20.937300    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b8e953d_030d_4edf_aa6a_232f0f55d1fa.slice/crio-008ab6b5beea4868d97617594aa7d074cb68cb405be85c59440fe2509561ff57 WatchSource:0}: Error finding container 008ab6b5beea4868d97617594aa7d074cb68cb405be85c59440fe2509561ff57: Status 404 returned error can't find the container with id 008ab6b5beea4868d97617594aa7d074cb68cb405be85c59440fe2509561ff57
Jul 24 18:16:21.073964 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:21.073868    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq" event={"ID":"6b8e953d-030d-4edf-aa6a-232f0f55d1fa","Type":"ContainerStarted","Data":"7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13"}
Jul 24 18:16:21.073964 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:21.073923    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq" event={"ID":"6b8e953d-030d-4edf-aa6a-232f0f55d1fa","Type":"ContainerStarted","Data":"008ab6b5beea4868d97617594aa7d074cb68cb405be85c59440fe2509561ff57"}
Jul 24 18:16:21.107194 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:21.107137    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq" podStartSLOduration=1.107117537 podStartE2EDuration="1.107117537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:16:21.104558427 +0000 UTC m=+2379.508499652" watchObservedRunningTime="2026-07-24 18:16:21.107117537 +0000 UTC m=+2379.511058762"
Jul 24 18:16:21.115242 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:21.115203    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:16:30.763037 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:30.763003    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:30.763567 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:30.763058    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:30.775487 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:30.775455    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:30.985029 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:30.984991    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:16:31.134717 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:31.134629    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:16:42.233384 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:42.233352    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:16:42.240876 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:42.240854    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:16:46.258943 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.258904    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"]
Jul 24 18:16:46.263810 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.263784    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.266296 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.266260    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-12361c\"/\"default-dockercfg-p5xf2\""
Jul 24 18:16:46.267112 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.267092    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-12361c\"/\"test-model-12361c-kserve-self-signed-certs\""
Jul 24 18:16:46.267209 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.267094    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-12361c\"/\"kube-root-ca.crt\""
Jul 24 18:16:46.267209 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.267151    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-12361c\"/\"openshift-service-ca.crt\""
Jul 24 18:16:46.274423 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.274392    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"]
Jul 24 18:16:46.374514 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.374474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tmp-dir\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.374514 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.374518    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-model-cache\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.374772 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.374615    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-dshm\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.374772 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.374670    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-home\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.374772 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.374693    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhx7f\" (UniqueName: \"kubernetes.io/projected/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-kube-api-access-nhx7f\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.374772 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.374744    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tls-certs\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476173 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tmp-dir\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476173 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476178    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-model-cache\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476431 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476220    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-dshm\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476431 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476257    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-home\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476431 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476305    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhx7f\" (UniqueName: \"kubernetes.io/projected/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-kube-api-access-nhx7f\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476431 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476326    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tls-certs\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476650 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476621    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tmp-dir\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476778 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476669    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-model-cache\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.476778 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.476693    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-home\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.478623 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.478599    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-dshm\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.478841 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.478827    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tls-certs\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.484149 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.484124    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhx7f\" (UniqueName: \"kubernetes.io/projected/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-kube-api-access-nhx7f\") pod \"test-model-12361c-kserve-695564f94f-rjx44\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") " pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.574776 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.574651    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:46.711860 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:46.711831    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"]
Jul 24 18:16:46.714613 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:16:46.714581    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeab2bdc9_d8d1_44b4_9635_64a5cbef59b2.slice/crio-07ac9dce20299e2ed4a743608601274fb78ae46602d5e0626944745d9ef04c83 WatchSource:0}: Error finding container 07ac9dce20299e2ed4a743608601274fb78ae46602d5e0626944745d9ef04c83: Status 404 returned error can't find the container with id 07ac9dce20299e2ed4a743608601274fb78ae46602d5e0626944745d9ef04c83
Jul 24 18:16:47.200650 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:47.200611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44" event={"ID":"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2","Type":"ContainerStarted","Data":"854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3"}
Jul 24 18:16:47.200897 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:47.200664    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44" event={"ID":"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2","Type":"ContainerStarted","Data":"07ac9dce20299e2ed4a743608601274fb78ae46602d5e0626944745d9ef04c83"}
Jul 24 18:16:47.221207 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:47.221151    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44" podStartSLOduration=1.221132475 podStartE2EDuration="1.221132475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:16:47.219943914 +0000 UTC m=+2405.623885143" watchObservedRunningTime="2026-07-24 18:16:47.221132475 +0000 UTC m=+2405.625073703"
Jul 24 18:16:50.271320 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:50.271285    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:16:56.575158 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:56.575110    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:56.575612 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:56.575178    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:56.587868 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:56.587838    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:16:57.259152 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:16:57.259114    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:17:05.165974 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:05.165889    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:17:33.692011 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:33.691966    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"]
Jul 24 18:17:34.747375 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:34.747336    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"]
Jul 24 18:17:38.461023 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:38.460982    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:17:48.714080 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:48.714027    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq" podUID="6b8e953d-030d-4edf-aa6a-232f0f55d1fa" containerName="main" containerID="cri-o://7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13" gracePeriod=45
Jul 24 18:17:48.976678 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:48.976603    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:17:49.063780 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.063745    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tmp-dir\") pod \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") "
Jul 24 18:17:49.063780 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.063784    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tls-certs\") pod \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") "
Jul 24 18:17:49.064045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.063818    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-dshm\") pod \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") "
Jul 24 18:17:49.064045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.063842    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-home\") pod \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") "
Jul 24 18:17:49.064045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.063866    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vjpj\" (UniqueName: \"kubernetes.io/projected/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-kube-api-access-2vjpj\") pod \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") "
Jul 24 18:17:49.064045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.063967    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-model-cache\") pod \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\" (UID: \"6b8e953d-030d-4edf-aa6a-232f0f55d1fa\") "
Jul 24 18:17:49.064249 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.064101    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6b8e953d-030d-4edf-aa6a-232f0f55d1fa" (UID: "6b8e953d-030d-4edf-aa6a-232f0f55d1fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:49.064249 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.064159    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-home" (OuterVolumeSpecName: "home") pod "6b8e953d-030d-4edf-aa6a-232f0f55d1fa" (UID: "6b8e953d-030d-4edf-aa6a-232f0f55d1fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:49.064440 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.064305    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:49.064440 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.064325    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:49.064440 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.064331    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "6b8e953d-030d-4edf-aa6a-232f0f55d1fa" (UID: "6b8e953d-030d-4edf-aa6a-232f0f55d1fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:49.066120 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.066098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-dshm" (OuterVolumeSpecName: "dshm") pod "6b8e953d-030d-4edf-aa6a-232f0f55d1fa" (UID: "6b8e953d-030d-4edf-aa6a-232f0f55d1fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:49.066201 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.066151    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6b8e953d-030d-4edf-aa6a-232f0f55d1fa" (UID: "6b8e953d-030d-4edf-aa6a-232f0f55d1fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:17:49.066241 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.066227    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-kube-api-access-2vjpj" (OuterVolumeSpecName: "kube-api-access-2vjpj") pod "6b8e953d-030d-4edf-aa6a-232f0f55d1fa" (UID: "6b8e953d-030d-4edf-aa6a-232f0f55d1fa"). InnerVolumeSpecName "kube-api-access-2vjpj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:17:49.165395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.165208    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:49.165395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.165241    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vjpj\" (UniqueName: \"kubernetes.io/projected/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-kube-api-access-2vjpj\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:49.165395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.165254    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:49.165395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.165299    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b8e953d-030d-4edf-aa6a-232f0f55d1fa-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:49.504494 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.498102    2575 generic.go:356] "Generic (PLEG): container finished" podID="6b8e953d-030d-4edf-aa6a-232f0f55d1fa" containerID="7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13" exitCode=0
Jul 24 18:17:49.504494 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.498240    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq" event={"ID":"6b8e953d-030d-4edf-aa6a-232f0f55d1fa","Type":"ContainerDied","Data":"7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13"}
Jul 24 18:17:49.504494 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.498306    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq" event={"ID":"6b8e953d-030d-4edf-aa6a-232f0f55d1fa","Type":"ContainerDied","Data":"008ab6b5beea4868d97617594aa7d074cb68cb405be85c59440fe2509561ff57"}
Jul 24 18:17:49.504494 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.498336    2575 scope.go:117] "RemoveContainer" containerID="7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13"
Jul 24 18:17:49.504494 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.498532    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"
Jul 24 18:17:49.517008 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.516986    2575 scope.go:117] "RemoveContainer" containerID="7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13"
Jul 24 18:17:49.517356 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:17:49.517331    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13\": container with ID starting with 7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13 not found: ID does not exist" containerID="7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13"
Jul 24 18:17:49.517419 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.517365    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13"} err="failed to get container status \"7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13\": rpc error: code = NotFound desc = could not find container \"7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13\": container with ID starting with 7c570ad0161cd82d41eeb6d4f2a9917a97b6f5d1542e2f15d7946ad24cc34d13 not found: ID does not exist"
Jul 24 18:17:49.525661 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.525627    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"]
Jul 24 18:17:49.529379 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.529345    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b3ffbc/test-model-b3ffbc-kserve-cff9fdbf-c5cqq"]
Jul 24 18:17:49.768054 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:49.767960    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44" podUID="eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" containerName="main" containerID="cri-o://854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3" gracePeriod=45
Jul 24 18:17:50.036183 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.036149    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:17:50.074310 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074251    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhx7f\" (UniqueName: \"kubernetes.io/projected/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-kube-api-access-nhx7f\") pod \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") "
Jul 24 18:17:50.074504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074340    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-dshm\") pod \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") "
Jul 24 18:17:50.074504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074380    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tls-certs\") pod \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") "
Jul 24 18:17:50.074504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074416    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tmp-dir\") pod \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") "
Jul 24 18:17:50.074504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074457    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-model-cache\") pod \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") "
Jul 24 18:17:50.074504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074476    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-home\") pod \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\" (UID: \"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2\") "
Jul 24 18:17:50.074857 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074765    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" (UID: "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:50.074857 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074791    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-home" (OuterVolumeSpecName: "home") pod "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" (UID: "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:50.074970 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.074895    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-model-cache" (OuterVolumeSpecName: "model-cache") pod "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" (UID: "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:50.076629 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.076595    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-dshm" (OuterVolumeSpecName: "dshm") pod "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" (UID: "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:17:50.076629 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.076600    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-kube-api-access-nhx7f" (OuterVolumeSpecName: "kube-api-access-nhx7f") pod "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" (UID: "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2"). InnerVolumeSpecName "kube-api-access-nhx7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:17:50.076829 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.076814    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" (UID: "eab2bdc9-d8d1-44b4-9635-64a5cbef59b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:17:50.138228 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.138182    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b8e953d-030d-4edf-aa6a-232f0f55d1fa" path="/var/lib/kubelet/pods/6b8e953d-030d-4edf-aa6a-232f0f55d1fa/volumes"
Jul 24 18:17:50.176104 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.176059    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhx7f\" (UniqueName: \"kubernetes.io/projected/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-kube-api-access-nhx7f\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:50.176104 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.176097    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:50.176104 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.176109    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:50.176399 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.176138    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:50.176399 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.176147    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:50.176399 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.176155    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:17:50.504806 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.504763    2575 generic.go:356] "Generic (PLEG): container finished" podID="eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" containerID="854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3" exitCode=0
Jul 24 18:17:50.504806 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.504813    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"
Jul 24 18:17:50.505054 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.504844    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44" event={"ID":"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2","Type":"ContainerDied","Data":"854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3"}
Jul 24 18:17:50.505054 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.504900    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44" event={"ID":"eab2bdc9-d8d1-44b4-9635-64a5cbef59b2","Type":"ContainerDied","Data":"07ac9dce20299e2ed4a743608601274fb78ae46602d5e0626944745d9ef04c83"}
Jul 24 18:17:50.505054 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.504918    2575 scope.go:117] "RemoveContainer" containerID="854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3"
Jul 24 18:17:50.513994 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.513972    2575 scope.go:117] "RemoveContainer" containerID="854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3"
Jul 24 18:17:50.514337 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:17:50.514315    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3\": container with ID starting with 854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3 not found: ID does not exist" containerID="854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3"
Jul 24 18:17:50.514414 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.514352    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3"} err="failed to get container status \"854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3\": rpc error: code = NotFound desc = could not find container \"854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3\": container with ID starting with 854375d30a12e30e05d739af347d5eff5096418a02b6442de3daf8f7c49861e3 not found: ID does not exist"
Jul 24 18:17:50.520027 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.519991    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"]
Jul 24 18:17:50.524912 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.524877    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-12361c/test-model-12361c-kserve-695564f94f-rjx44"]
Jul 24 18:17:50.981804 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:50.981770    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:17:52.138205 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:17:52.138165    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" path="/var/lib/kubelet/pods/eab2bdc9-d8d1-44b4-9635-64a5cbef59b2/volumes"
Jul 24 18:18:10.458651 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:18:10.458607    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:18:29.803114 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:18:29.803080    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:18:43.277298 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:18:43.275949    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:19:00.663836 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:19:00.663793    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:19:30.469834 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:19:30.469789    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:19:45.160539 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:19:45.160472    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:19:53.891636 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:19:53.891598    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:20:06.960985 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.960898    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"]
Jul 24 18:20:06.961480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.961313    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" containerName="main"
Jul 24 18:20:06.961480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.961325    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" containerName="main"
Jul 24 18:20:06.961480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.961346    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8e953d-030d-4edf-aa6a-232f0f55d1fa" containerName="main"
Jul 24 18:20:06.961480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.961351    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8e953d-030d-4edf-aa6a-232f0f55d1fa" containerName="main"
Jul 24 18:20:06.961480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.961401    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="eab2bdc9-d8d1-44b4-9635-64a5cbef59b2" containerName="main"
Jul 24 18:20:06.961480 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.961410    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8e953d-030d-4edf-aa6a-232f0f55d1fa" containerName="main"
Jul 24 18:20:06.964504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.964482    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:06.967956 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.967915    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"kube-root-ca.crt\""
Jul 24 18:20:06.968099 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.967976    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"openshift-service-ca.crt\""
Jul 24 18:20:06.968099 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.967983    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"auth-test-model-100cff-kserve-self-signed-certs\""
Jul 24 18:20:06.968099 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.967922    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"default-dockercfg-5n5v7\""
Jul 24 18:20:06.974119 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:06.974085    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"]
Jul 24 18:20:07.018787 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.018752    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-home\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.018968 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.018804    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-model-cache\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.018968 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.018864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-dshm\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.018968 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.018892    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86x5g\" (UniqueName: \"kubernetes.io/projected/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-kube-api-access-86x5g\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.018968 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.018919    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tls-certs\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.018968 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.018940    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tmp-dir\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120109 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120060    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-dshm\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120109 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120104    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86x5g\" (UniqueName: \"kubernetes.io/projected/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-kube-api-access-86x5g\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120425 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120129    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tls-certs\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120425 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120152    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tmp-dir\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120425 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120220    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-home\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120425 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120295    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-model-cache\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120652 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120622    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-home\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120709 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120663    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tmp-dir\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.120709 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.120686    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-model-cache\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.122504 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.122482    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-dshm\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.122646 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.122628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tls-certs\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.129106 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.129076    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86x5g\" (UniqueName: \"kubernetes.io/projected/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-kube-api-access-86x5g\") pod \"auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") " pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.274520 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.274421    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:07.417106 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:07.417076    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"]
Jul 24 18:20:07.419460 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:20:07.419429    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c7af2af_af5c_4cd2_b4b1_502ba663ea2b.slice/crio-6e47b2c2335ec9be6e34606530b83a88d4776de1edc9055019758182548b3e4e WatchSource:0}: Error finding container 6e47b2c2335ec9be6e34606530b83a88d4776de1edc9055019758182548b3e4e: Status 404 returned error can't find the container with id 6e47b2c2335ec9be6e34606530b83a88d4776de1edc9055019758182548b3e4e
Jul 24 18:20:08.176254 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:08.176211    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb" event={"ID":"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b","Type":"ContainerStarted","Data":"1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c"}
Jul 24 18:20:08.176679 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:08.176280    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb" event={"ID":"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b","Type":"ContainerStarted","Data":"6e47b2c2335ec9be6e34606530b83a88d4776de1edc9055019758182548b3e4e"}
Jul 24 18:20:08.202340 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:08.202284    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb" podStartSLOduration=2.202246784 podStartE2EDuration="2.202246784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:20:08.200245378 +0000 UTC m=+2606.604186607" watchObservedRunningTime="2026-07-24 18:20:08.202246784 +0000 UTC m=+2606.606188014"
Jul 24 18:20:10.277427 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:10.277382    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:20:17.275126 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:17.275086    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:17.275647 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:17.275163    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:17.288174 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:17.288141    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:18.238747 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:18.238712    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:20:36.740345 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.740301    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"]
Jul 24 18:20:36.743788 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.743764    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.747415 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.747372    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"auth-test-model-c4d732-kserve-self-signed-certs\""
Jul 24 18:20:36.747608 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.747419    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"default-dockercfg-czpdh\""
Jul 24 18:20:36.747608 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.747384    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"kube-root-ca.crt\""
Jul 24 18:20:36.747721 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.747705    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"openshift-service-ca.crt\""
Jul 24 18:20:36.755085 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.755048    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"]
Jul 24 18:20:36.796445 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.796398    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-model-cache\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.796445 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.796452    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-tmp-dir\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.796680 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.796542    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04abf82c-ad55-48ce-a27b-a01532b2f3af-tls-certs\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.796680 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.796595    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-home\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.796680 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.796620    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-dshm\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.796680 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.796643    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2v2t\" (UniqueName: \"kubernetes.io/projected/04abf82c-ad55-48ce-a27b-a01532b2f3af-kube-api-access-g2v2t\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.897989 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.897938    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-home\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.897989 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.897984    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-dshm\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898001    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2v2t\" (UniqueName: \"kubernetes.io/projected/04abf82c-ad55-48ce-a27b-a01532b2f3af-kube-api-access-g2v2t\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898049    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-model-cache\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898086    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-tmp-dir\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04abf82c-ad55-48ce-a27b-a01532b2f3af-tls-certs\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898559 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898536    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-model-cache\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898610 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898540    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-home\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.898610 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.898584    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-tmp-dir\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.900413 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.900382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-dshm\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.900596 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.900577    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04abf82c-ad55-48ce-a27b-a01532b2f3af-tls-certs\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:36.908496 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:36.908466    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2v2t\" (UniqueName: \"kubernetes.io/projected/04abf82c-ad55-48ce-a27b-a01532b2f3af-kube-api-access-g2v2t\") pod \"auth-test-model-c4d732-kserve-5869978955-mmnrr\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") " pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:37.054950 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:37.054845    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:37.198938 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:37.198813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"]
Jul 24 18:20:37.201852 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:20:37.201821    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04abf82c_ad55_48ce_a27b_a01532b2f3af.slice/crio-55c9a82ea62ab23ac3145dc0853b4f68373f718b84868e9b36aa4c0bb1600c69 WatchSource:0}: Error finding container 55c9a82ea62ab23ac3145dc0853b4f68373f718b84868e9b36aa4c0bb1600c69: Status 404 returned error can't find the container with id 55c9a82ea62ab23ac3145dc0853b4f68373f718b84868e9b36aa4c0bb1600c69
Jul 24 18:20:37.320903 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:37.320790    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr" event={"ID":"04abf82c-ad55-48ce-a27b-a01532b2f3af","Type":"ContainerStarted","Data":"1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72"}
Jul 24 18:20:37.320903 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:37.320841    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr" event={"ID":"04abf82c-ad55-48ce-a27b-a01532b2f3af","Type":"ContainerStarted","Data":"55c9a82ea62ab23ac3145dc0853b4f68373f718b84868e9b36aa4c0bb1600c69"}
Jul 24 18:20:37.338008 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:37.337941    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr" podStartSLOduration=1.337918784 podStartE2EDuration="1.337918784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:20:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:20:37.337427752 +0000 UTC m=+2635.741368980" watchObservedRunningTime="2026-07-24 18:20:37.337918784 +0000 UTC m=+2635.741860015"
Jul 24 18:20:40.162855 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:40.162813    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:20:47.055259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:47.055214    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:47.055259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:47.055297    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:47.068029 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:47.068001    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:47.382323 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:47.382222    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:20:59.858442 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:20:59.858402    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:21:18.958030 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:21:18.957988    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:21:35.561200 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:21:35.561110    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:21:42.264025 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:21:42.263986    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:21:42.272257 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:21:42.272227    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:21:54.763531 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:21:54.763493    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:22:12.560077 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:22:12.560033    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:22:32.363535 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:22:32.363499    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:22:51.660439 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:22:51.660399    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:22:56.758227 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:22:56.758188    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"]
Jul 24 18:22:57.736651 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:22:57.736617    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"]
Jul 24 18:23:08.860533 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:08.860491    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:23:11.778899 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:11.778829    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb" podUID="0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" containerName="main" containerID="cri-o://1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c" gracePeriod=45
Jul 24 18:23:12.031731 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.031665    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:23:12.075575 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.075531    2575 generic.go:356] "Generic (PLEG): container finished" podID="0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" containerID="1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c" exitCode=0
Jul 24 18:23:12.075575 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.075575    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"
Jul 24 18:23:12.075843 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.075611    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb" event={"ID":"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b","Type":"ContainerDied","Data":"1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c"}
Jul 24 18:23:12.075843 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.075651    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb" event={"ID":"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b","Type":"ContainerDied","Data":"6e47b2c2335ec9be6e34606530b83a88d4776de1edc9055019758182548b3e4e"}
Jul 24 18:23:12.075843 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.075668    2575 scope.go:117] "RemoveContainer" containerID="1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c"
Jul 24 18:23:12.084695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.084675    2575 scope.go:117] "RemoveContainer" containerID="1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c"
Jul 24 18:23:12.084977 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:23:12.084957    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c\": container with ID starting with 1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c not found: ID does not exist" containerID="1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c"
Jul 24 18:23:12.085032 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.084988    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c"} err="failed to get container status \"1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c\": rpc error: code = NotFound desc = could not find container \"1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c\": container with ID starting with 1e3455e0b5f0b21f5d6d3488043db3def298f674c760cdc8ad6a3fe872a8b14c not found: ID does not exist"
Jul 24 18:23:12.163785 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.163745    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-model-cache\") pod \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") "
Jul 24 18:23:12.163785 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.163788    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86x5g\" (UniqueName: \"kubernetes.io/projected/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-kube-api-access-86x5g\") pod \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") "
Jul 24 18:23:12.164043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.163847    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tmp-dir\") pod \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") "
Jul 24 18:23:12.164043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.163874    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-dshm\") pod \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") "
Jul 24 18:23:12.164043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.163896    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tls-certs\") pod \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") "
Jul 24 18:23:12.164043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.163967    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-home\") pod \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\" (UID: \"0c7af2af-af5c-4cd2-b4b1-502ba663ea2b\") "
Jul 24 18:23:12.164254 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.164098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" (UID: "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:12.164254 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.164166    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" (UID: "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:12.164390 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.164334    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-home" (OuterVolumeSpecName: "home") pod "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" (UID: "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:12.164390 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.164339    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:12.164390 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.164372    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:12.166176 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.166151    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-dshm" (OuterVolumeSpecName: "dshm") pod "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" (UID: "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:12.166176 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.166164    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" (UID: "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:23:12.166310 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.166179    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-kube-api-access-86x5g" (OuterVolumeSpecName: "kube-api-access-86x5g") pod "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" (UID: "0c7af2af-af5c-4cd2-b4b1-502ba663ea2b"). InnerVolumeSpecName "kube-api-access-86x5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:23:12.265695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.265650    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:12.265695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.265690    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86x5g\" (UniqueName: \"kubernetes.io/projected/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-kube-api-access-86x5g\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:12.265695 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.265704    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:12.265945 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.265715    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:12.397349 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.397316    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"]
Jul 24 18:23:12.404376 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.401901    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/auth-test-model-100cff-kserve-69cf9cd7dc-s9vmb"]
Jul 24 18:23:12.756737 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:12.756666    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr" podUID="04abf82c-ad55-48ce-a27b-a01532b2f3af" containerName="main" containerID="cri-o://1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72" gracePeriod=45
Jul 24 18:23:13.005933 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.005903    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:23:13.082817 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.082719    2575 generic.go:356] "Generic (PLEG): container finished" podID="04abf82c-ad55-48ce-a27b-a01532b2f3af" containerID="1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72" exitCode=0
Jul 24 18:23:13.082817 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.082769    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"
Jul 24 18:23:13.082817 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.082798    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr" event={"ID":"04abf82c-ad55-48ce-a27b-a01532b2f3af","Type":"ContainerDied","Data":"1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72"}
Jul 24 18:23:13.083076 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.082835    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr" event={"ID":"04abf82c-ad55-48ce-a27b-a01532b2f3af","Type":"ContainerDied","Data":"55c9a82ea62ab23ac3145dc0853b4f68373f718b84868e9b36aa4c0bb1600c69"}
Jul 24 18:23:13.083076 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.082855    2575 scope.go:117] "RemoveContainer" containerID="1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72"
Jul 24 18:23:13.091307 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.091288    2575 scope.go:117] "RemoveContainer" containerID="1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72"
Jul 24 18:23:13.091638 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:23:13.091619    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72\": container with ID starting with 1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72 not found: ID does not exist" containerID="1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72"
Jul 24 18:23:13.091705 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.091654    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72"} err="failed to get container status \"1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72\": rpc error: code = NotFound desc = could not find container \"1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72\": container with ID starting with 1235bd89b95caccacc6f0c612183ef040df8040d507c4853da8e2d3cf3ac5a72 not found: ID does not exist"
Jul 24 18:23:13.174389 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174348    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-tmp-dir\") pod \"04abf82c-ad55-48ce-a27b-a01532b2f3af\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") "
Jul 24 18:23:13.174549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174406    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-dshm\") pod \"04abf82c-ad55-48ce-a27b-a01532b2f3af\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") "
Jul 24 18:23:13.174549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174429    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-model-cache\") pod \"04abf82c-ad55-48ce-a27b-a01532b2f3af\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") "
Jul 24 18:23:13.174549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174474    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04abf82c-ad55-48ce-a27b-a01532b2f3af-tls-certs\") pod \"04abf82c-ad55-48ce-a27b-a01532b2f3af\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") "
Jul 24 18:23:13.174549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174497    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2v2t\" (UniqueName: \"kubernetes.io/projected/04abf82c-ad55-48ce-a27b-a01532b2f3af-kube-api-access-g2v2t\") pod \"04abf82c-ad55-48ce-a27b-a01532b2f3af\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") "
Jul 24 18:23:13.174549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174550    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-home\") pod \"04abf82c-ad55-48ce-a27b-a01532b2f3af\" (UID: \"04abf82c-ad55-48ce-a27b-a01532b2f3af\") "
Jul 24 18:23:13.174836 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174669    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "04abf82c-ad55-48ce-a27b-a01532b2f3af" (UID: "04abf82c-ad55-48ce-a27b-a01532b2f3af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:13.174836 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174807    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-model-cache" (OuterVolumeSpecName: "model-cache") pod "04abf82c-ad55-48ce-a27b-a01532b2f3af" (UID: "04abf82c-ad55-48ce-a27b-a01532b2f3af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:13.174939 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174900    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-home" (OuterVolumeSpecName: "home") pod "04abf82c-ad55-48ce-a27b-a01532b2f3af" (UID: "04abf82c-ad55-48ce-a27b-a01532b2f3af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:13.174939 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174927    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:13.175022 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.174948    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:13.176860 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.176839    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-dshm" (OuterVolumeSpecName: "dshm") pod "04abf82c-ad55-48ce-a27b-a01532b2f3af" (UID: "04abf82c-ad55-48ce-a27b-a01532b2f3af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:23:13.176938 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.176893    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04abf82c-ad55-48ce-a27b-a01532b2f3af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "04abf82c-ad55-48ce-a27b-a01532b2f3af" (UID: "04abf82c-ad55-48ce-a27b-a01532b2f3af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:23:13.176938 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.176926    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04abf82c-ad55-48ce-a27b-a01532b2f3af-kube-api-access-g2v2t" (OuterVolumeSpecName: "kube-api-access-g2v2t") pod "04abf82c-ad55-48ce-a27b-a01532b2f3af" (UID: "04abf82c-ad55-48ce-a27b-a01532b2f3af"). InnerVolumeSpecName "kube-api-access-g2v2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:23:13.275813 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.275776    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:13.275813 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.275808    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04abf82c-ad55-48ce-a27b-a01532b2f3af-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:13.275813 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.275818    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2v2t\" (UniqueName: \"kubernetes.io/projected/04abf82c-ad55-48ce-a27b-a01532b2f3af-kube-api-access-g2v2t\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:13.276057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.275828    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04abf82c-ad55-48ce-a27b-a01532b2f3af-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:23:13.404876 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.404841    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"]
Jul 24 18:23:13.409386 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:13.409362    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/auth-test-model-c4d732-kserve-5869978955-mmnrr"]
Jul 24 18:23:14.138942 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.138910    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04abf82c-ad55-48ce-a27b-a01532b2f3af" path="/var/lib/kubelet/pods/04abf82c-ad55-48ce-a27b-a01532b2f3af/volumes"
Jul 24 18:23:14.139536 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.139514    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" path="/var/lib/kubelet/pods/0c7af2af-af5c-4cd2-b4b1-502ba663ea2b/volumes"
Jul 24 18:23:14.162807 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.162766    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"]
Jul 24 18:23:14.163221 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.163202    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" containerName="main"
Jul 24 18:23:14.163331 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.163222    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" containerName="main"
Jul 24 18:23:14.163331 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.163290    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04abf82c-ad55-48ce-a27b-a01532b2f3af" containerName="main"
Jul 24 18:23:14.163331 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.163300    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="04abf82c-ad55-48ce-a27b-a01532b2f3af" containerName="main"
Jul 24 18:23:14.163485 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.163409    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c7af2af-af5c-4cd2-b4b1-502ba663ea2b" containerName="main"
Jul 24 18:23:14.163485 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.163427    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="04abf82c-ad55-48ce-a27b-a01532b2f3af" containerName="main"
Jul 24 18:23:14.167659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.167635    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.170455 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.170432    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"openshift-service-ca.crt\""
Jul 24 18:23:14.170567 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.170536    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"default-dockercfg-5n5v7\""
Jul 24 18:23:14.170989 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.170970    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"kube-root-ca.crt\""
Jul 24 18:23:14.171088 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.171030    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"sub-test-model-100cff-kserve-self-signed-certs\""
Jul 24 18:23:14.176845 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.176820    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"]
Jul 24 18:23:14.284378 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.284337    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-home\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.284378 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.284378    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tls-certs\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.284614 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.284417    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-dshm\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.284614 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.284453    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tmp-dir\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.284614 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.284476    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c22x9\" (UniqueName: \"kubernetes.io/projected/f544d2c3-ffa0-4158-82f3-4539a5424eb7-kube-api-access-c22x9\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.284614 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.284543    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-model-cache\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385179 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385139    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-dshm\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385179 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385185    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tmp-dir\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385430 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c22x9\" (UniqueName: \"kubernetes.io/projected/f544d2c3-ffa0-4158-82f3-4539a5424eb7-kube-api-access-c22x9\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385430 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385303    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-model-cache\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385430 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-home\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385430 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385401    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tls-certs\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385757 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385723    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tmp-dir\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385757 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385743    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-home\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.385872 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.385784    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-model-cache\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.387533 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.387513    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-dshm\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.387805 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.387787    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tls-certs\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.394831 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.394765    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c22x9\" (UniqueName: \"kubernetes.io/projected/f544d2c3-ffa0-4158-82f3-4539a5424eb7-kube-api-access-c22x9\") pod \"sub-test-model-100cff-kserve-8d74b5f9f-f7nxq\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") " pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.477436 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.477390    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:14.615393 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:14.615208    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"]
Jul 24 18:23:14.618279 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:23:14.618233    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf544d2c3_ffa0_4158_82f3_4539a5424eb7.slice/crio-2d3a6cc166f5097290de1fe174850bba9d8cd52ba0a5c79e521ccba7362a93d0 WatchSource:0}: Error finding container 2d3a6cc166f5097290de1fe174850bba9d8cd52ba0a5c79e521ccba7362a93d0: Status 404 returned error can't find the container with id 2d3a6cc166f5097290de1fe174850bba9d8cd52ba0a5c79e521ccba7362a93d0
Jul 24 18:23:15.097719 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:15.097683    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq" event={"ID":"f544d2c3-ffa0-4158-82f3-4539a5424eb7","Type":"ContainerStarted","Data":"e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a"}
Jul 24 18:23:15.097719 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:15.097723    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq" event={"ID":"f544d2c3-ffa0-4158-82f3-4539a5424eb7","Type":"ContainerStarted","Data":"2d3a6cc166f5097290de1fe174850bba9d8cd52ba0a5c79e521ccba7362a93d0"}
Jul 24 18:23:15.115777 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:15.115715    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq" podStartSLOduration=1.115698067 podStartE2EDuration="1.115698067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:23:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:23:15.113935979 +0000 UTC m=+2793.517877210" watchObservedRunningTime="2026-07-24 18:23:15.115698067 +0000 UTC m=+2793.519639295"
Jul 24 18:23:19.957957 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:19.957917    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:23:24.477506 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:24.477459    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:24.480063 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:24.477536    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:24.490546 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:24.490510    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:23:25.163192 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:23:25.163160    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:24:06.754412 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.754366    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"]
Jul 24 18:24:06.758017 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.757993    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.760394 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.760360    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"kube-root-ca.crt\""
Jul 24 18:24:06.760394 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.760391    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"openshift-service-ca.crt\""
Jul 24 18:24:06.761183 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.761165    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"sub-test-model-c4d732-kserve-self-signed-certs\""
Jul 24 18:24:06.761319 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.761255    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"default-dockercfg-czpdh\""
Jul 24 18:24:06.769133 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.769108    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"]
Jul 24 18:24:06.858129 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.858088    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-model-cache\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.858328 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.858143    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d345084-fc30-4bae-a511-092c143e5bf8-tls-certs\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.858328 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.858260    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-dshm\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.858328 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.858308    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5kvq\" (UniqueName: \"kubernetes.io/projected/2d345084-fc30-4bae-a511-092c143e5bf8-kube-api-access-t5kvq\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.858446 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.858382    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-tmp-dir\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.858446 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.858410    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-home\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959311 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959243    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-model-cache\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959311 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d345084-fc30-4bae-a511-092c143e5bf8-tls-certs\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959380    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-dshm\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959402    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5kvq\" (UniqueName: \"kubernetes.io/projected/2d345084-fc30-4bae-a511-092c143e5bf8-kube-api-access-t5kvq\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959431    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-tmp-dir\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959549 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959447    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-home\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959719 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959681    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-model-cache\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959776 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959756    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-tmp-dir\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.959833 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.959774    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-home\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.961684 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.961657    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-dshm\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.961974 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.961952    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d345084-fc30-4bae-a511-092c143e5bf8-tls-certs\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:06.969122 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:06.969091    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5kvq\" (UniqueName: \"kubernetes.io/projected/2d345084-fc30-4bae-a511-092c143e5bf8-kube-api-access-t5kvq\") pod \"sub-test-model-c4d732-kserve-6999c68b4f-rfthb\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") " pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:07.068296 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:07.068174    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:07.207472 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:07.207444    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"]
Jul 24 18:24:07.209670 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:24:07.209645    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d345084_fc30_4bae_a511_092c143e5bf8.slice/crio-7e43f1c178f8e78a044070e83040e6b011c62a7f4623ec106138cefb92b252ea WatchSource:0}: Error finding container 7e43f1c178f8e78a044070e83040e6b011c62a7f4623ec106138cefb92b252ea: Status 404 returned error can't find the container with id 7e43f1c178f8e78a044070e83040e6b011c62a7f4623ec106138cefb92b252ea
Jul 24 18:24:07.355146 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:07.355048    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb" event={"ID":"2d345084-fc30-4bae-a511-092c143e5bf8","Type":"ContainerStarted","Data":"69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8"}
Jul 24 18:24:07.355146 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:07.355091    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb" event={"ID":"2d345084-fc30-4bae-a511-092c143e5bf8","Type":"ContainerStarted","Data":"7e43f1c178f8e78a044070e83040e6b011c62a7f4623ec106138cefb92b252ea"}
Jul 24 18:24:07.373517 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:07.373466    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb" podStartSLOduration=1.37344891 podStartE2EDuration="1.37344891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:24:07.372575131 +0000 UTC m=+2845.776516361" watchObservedRunningTime="2026-07-24 18:24:07.37344891 +0000 UTC m=+2845.777390138"
Jul 24 18:24:10.267943 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:10.267904    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:24:17.068339 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:17.068293    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:17.068936 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:17.068354    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:17.080890 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:17.080859    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:17.416490 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:17.416398    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:25.267481 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:25.267404    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:24:42.147047 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:42.146955    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"]
Jul 24 18:24:43.063004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:43.062967    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"]
Jul 24 18:24:47.751503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:47.751463    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:24:57.166932 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.166887    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq" podUID="f544d2c3-ffa0-4158-82f3-4539a5424eb7" containerName="main" containerID="cri-o://e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a" gracePeriod=45
Jul 24 18:24:57.416256 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.416224    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:24:57.443040 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443007    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-dshm\") pod \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") "
Jul 24 18:24:57.443040 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443044    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tls-certs\") pod \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") "
Jul 24 18:24:57.443365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443107    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-home\") pod \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") "
Jul 24 18:24:57.443365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443134    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-model-cache\") pod \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") "
Jul 24 18:24:57.443365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443154    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tmp-dir\") pod \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") "
Jul 24 18:24:57.443365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443199    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c22x9\" (UniqueName: \"kubernetes.io/projected/f544d2c3-ffa0-4158-82f3-4539a5424eb7-kube-api-access-c22x9\") pod \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\" (UID: \"f544d2c3-ffa0-4158-82f3-4539a5424eb7\") "
Jul 24 18:24:57.443586 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443370    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-model-cache" (OuterVolumeSpecName: "model-cache") pod "f544d2c3-ffa0-4158-82f3-4539a5424eb7" (UID: "f544d2c3-ffa0-4158-82f3-4539a5424eb7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:57.443586 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443388    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-home" (OuterVolumeSpecName: "home") pod "f544d2c3-ffa0-4158-82f3-4539a5424eb7" (UID: "f544d2c3-ffa0-4158-82f3-4539a5424eb7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:57.443586 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443532    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:57.443586 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443552    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:57.443586 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.443549    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f544d2c3-ffa0-4158-82f3-4539a5424eb7" (UID: "f544d2c3-ffa0-4158-82f3-4539a5424eb7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:57.445279 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.445238    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-dshm" (OuterVolumeSpecName: "dshm") pod "f544d2c3-ffa0-4158-82f3-4539a5424eb7" (UID: "f544d2c3-ffa0-4158-82f3-4539a5424eb7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:57.445279 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.445250    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f544d2c3-ffa0-4158-82f3-4539a5424eb7" (UID: "f544d2c3-ffa0-4158-82f3-4539a5424eb7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:24:57.445465 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.445383    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f544d2c3-ffa0-4158-82f3-4539a5424eb7-kube-api-access-c22x9" (OuterVolumeSpecName: "kube-api-access-c22x9") pod "f544d2c3-ffa0-4158-82f3-4539a5424eb7" (UID: "f544d2c3-ffa0-4158-82f3-4539a5424eb7"). InnerVolumeSpecName "kube-api-access-c22x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:24:57.544489 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.544451    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:57.544489 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.544482    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:57.544489 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.544492    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f544d2c3-ffa0-4158-82f3-4539a5424eb7-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:57.544732 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.544502    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c22x9\" (UniqueName: \"kubernetes.io/projected/f544d2c3-ffa0-4158-82f3-4539a5424eb7-kube-api-access-c22x9\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:57.601797 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.601768    2575 generic.go:356] "Generic (PLEG): container finished" podID="f544d2c3-ffa0-4158-82f3-4539a5424eb7" containerID="e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a" exitCode=0
Jul 24 18:24:57.601981 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.601805    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq" event={"ID":"f544d2c3-ffa0-4158-82f3-4539a5424eb7","Type":"ContainerDied","Data":"e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a"}
Jul 24 18:24:57.601981 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.601815    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"
Jul 24 18:24:57.601981 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.601833    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq" event={"ID":"f544d2c3-ffa0-4158-82f3-4539a5424eb7","Type":"ContainerDied","Data":"2d3a6cc166f5097290de1fe174850bba9d8cd52ba0a5c79e521ccba7362a93d0"}
Jul 24 18:24:57.601981 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.601852    2575 scope.go:117] "RemoveContainer" containerID="e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a"
Jul 24 18:24:57.611918 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.611895    2575 scope.go:117] "RemoveContainer" containerID="e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a"
Jul 24 18:24:57.612281 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:24:57.612239    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a\": container with ID starting with e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a not found: ID does not exist" containerID="e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a"
Jul 24 18:24:57.612370 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.612289    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a"} err="failed to get container status \"e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a\": rpc error: code = NotFound desc = could not find container \"e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a\": container with ID starting with e974fa7570fa0a79fae0792bb09630326d581e564b132cf22a6db79335909e3a not found: ID does not exist"
Jul 24 18:24:57.623434 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.623398    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"]
Jul 24 18:24:57.627827 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:57.627797    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/sub-test-model-100cff-kserve-8d74b5f9f-f7nxq"]
Jul 24 18:24:58.082772 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.082725    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb" podUID="2d345084-fc30-4bae-a511-092c143e5bf8" containerName="main" containerID="cri-o://69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8" gracePeriod=45
Jul 24 18:24:58.137961 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.137922    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f544d2c3-ffa0-4158-82f3-4539a5424eb7" path="/var/lib/kubelet/pods/f544d2c3-ffa0-4158-82f3-4539a5424eb7/volumes"
Jul 24 18:24:58.341745 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.341681    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:58.453752 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.453708    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d345084-fc30-4bae-a511-092c143e5bf8-tls-certs\") pod \"2d345084-fc30-4bae-a511-092c143e5bf8\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") "
Jul 24 18:24:58.453956 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.453774    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-tmp-dir\") pod \"2d345084-fc30-4bae-a511-092c143e5bf8\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") "
Jul 24 18:24:58.453956 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.453826    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5kvq\" (UniqueName: \"kubernetes.io/projected/2d345084-fc30-4bae-a511-092c143e5bf8-kube-api-access-t5kvq\") pod \"2d345084-fc30-4bae-a511-092c143e5bf8\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") "
Jul 24 18:24:58.453956 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.453923    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-model-cache\") pod \"2d345084-fc30-4bae-a511-092c143e5bf8\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") "
Jul 24 18:24:58.454129 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.453991    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-home\") pod \"2d345084-fc30-4bae-a511-092c143e5bf8\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") "
Jul 24 18:24:58.454129 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454014    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-dshm\") pod \"2d345084-fc30-4bae-a511-092c143e5bf8\" (UID: \"2d345084-fc30-4bae-a511-092c143e5bf8\") "
Jul 24 18:24:58.454129 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454040    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d345084-fc30-4bae-a511-092c143e5bf8" (UID: "2d345084-fc30-4bae-a511-092c143e5bf8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:58.454260 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454168    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d345084-fc30-4bae-a511-092c143e5bf8" (UID: "2d345084-fc30-4bae-a511-092c143e5bf8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:58.454335 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454304    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-home" (OuterVolumeSpecName: "home") pod "2d345084-fc30-4bae-a511-092c143e5bf8" (UID: "2d345084-fc30-4bae-a511-092c143e5bf8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:58.454488 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454466    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:58.454525 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454496    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:58.454525 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.454510    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:58.456080 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.456054    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-dshm" (OuterVolumeSpecName: "dshm") pod "2d345084-fc30-4bae-a511-092c143e5bf8" (UID: "2d345084-fc30-4bae-a511-092c143e5bf8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:24:58.456080 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.456066    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d345084-fc30-4bae-a511-092c143e5bf8-kube-api-access-t5kvq" (OuterVolumeSpecName: "kube-api-access-t5kvq") pod "2d345084-fc30-4bae-a511-092c143e5bf8" (UID: "2d345084-fc30-4bae-a511-092c143e5bf8"). InnerVolumeSpecName "kube-api-access-t5kvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:24:58.456229 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.456054    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d345084-fc30-4bae-a511-092c143e5bf8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d345084-fc30-4bae-a511-092c143e5bf8" (UID: "2d345084-fc30-4bae-a511-092c143e5bf8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:24:58.555555 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.555517    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d345084-fc30-4bae-a511-092c143e5bf8-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:58.555555 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.555550    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5kvq\" (UniqueName: \"kubernetes.io/projected/2d345084-fc30-4bae-a511-092c143e5bf8-kube-api-access-t5kvq\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:58.555555 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.555560    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d345084-fc30-4bae-a511-092c143e5bf8-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:24:58.607519 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.607434    2575 generic.go:356] "Generic (PLEG): container finished" podID="2d345084-fc30-4bae-a511-092c143e5bf8" containerID="69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8" exitCode=0
Jul 24 18:24:58.607519 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.607486    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"
Jul 24 18:24:58.607725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.607522    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb" event={"ID":"2d345084-fc30-4bae-a511-092c143e5bf8","Type":"ContainerDied","Data":"69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8"}
Jul 24 18:24:58.607725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.607577    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb" event={"ID":"2d345084-fc30-4bae-a511-092c143e5bf8","Type":"ContainerDied","Data":"7e43f1c178f8e78a044070e83040e6b011c62a7f4623ec106138cefb92b252ea"}
Jul 24 18:24:58.607725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.607604    2575 scope.go:117] "RemoveContainer" containerID="69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8"
Jul 24 18:24:58.617014 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.616996    2575 scope.go:117] "RemoveContainer" containerID="69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8"
Jul 24 18:24:58.617329 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:24:58.617308    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8\": container with ID starting with 69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8 not found: ID does not exist" containerID="69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8"
Jul 24 18:24:58.617377 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.617338    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8"} err="failed to get container status \"69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8\": rpc error: code = NotFound desc = could not find container \"69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8\": container with ID starting with 69a423e5bcf2538cbcad2be0618658b76f6e3165525ce69ae5c2855650c1fce8 not found: ID does not exist"
Jul 24 18:24:58.628717 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.628689    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"]
Jul 24 18:24:58.632420 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:24:58.632398    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/sub-test-model-c4d732-kserve-6999c68b4f-rfthb"]
Jul 24 18:25:00.137992 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:00.137954    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d345084-fc30-4bae-a511-092c143e5bf8" path="/var/lib/kubelet/pods/2d345084-fc30-4bae-a511-092c143e5bf8/volumes"
Jul 24 18:25:03.767525 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767482    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"]
Jul 24 18:25:03.767930 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767862    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f544d2c3-ffa0-4158-82f3-4539a5424eb7" containerName="main"
Jul 24 18:25:03.767930 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767873    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f544d2c3-ffa0-4158-82f3-4539a5424eb7" containerName="main"
Jul 24 18:25:03.767930 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767886    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d345084-fc30-4bae-a511-092c143e5bf8" containerName="main"
Jul 24 18:25:03.767930 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767892    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d345084-fc30-4bae-a511-092c143e5bf8" containerName="main"
Jul 24 18:25:03.768072 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767942    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f544d2c3-ffa0-4158-82f3-4539a5424eb7" containerName="main"
Jul 24 18:25:03.768072 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.767951    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d345084-fc30-4bae-a511-092c143e5bf8" containerName="main"
Jul 24 18:25:03.772101 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.772070    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.779835 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.779798    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"kube-root-ca.crt\""
Jul 24 18:25:03.781794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.781021    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"openshift-service-ca.crt\""
Jul 24 18:25:03.781794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.781146    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"rate-test-model-100cff-kserve-self-signed-certs\""
Jul 24 18:25:03.783457 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.783173    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-100cff\"/\"default-dockercfg-5n5v7\""
Jul 24 18:25:03.785043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.785014    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"]
Jul 24 18:25:03.805088 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.805048    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-dshm\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.805088 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.805092    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tmp-dir\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.805365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.805119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpht9\" (UniqueName: \"kubernetes.io/projected/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-kube-api-access-kpht9\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.805365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.805238    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-model-cache\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.805365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.805336    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tls-certs\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.805365 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.805358    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-home\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.906765 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.906699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-model-cache\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.906765 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.906783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tls-certs\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907046 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.906804    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-home\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907046 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.906846    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-dshm\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907046 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.906860    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tmp-dir\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907046 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.906876    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpht9\" (UniqueName: \"kubernetes.io/projected/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-kube-api-access-kpht9\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907349 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.907325    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-home\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907417 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.907368    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-model-cache\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.907417 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.907382    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tmp-dir\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.909246 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.909225    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-dshm\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.909624 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.909600    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tls-certs\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:03.915569 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:03.915531    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpht9\" (UniqueName: \"kubernetes.io/projected/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-kube-api-access-kpht9\") pod \"rate-test-model-100cff-kserve-7488bb7bc5-l7mjs\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") " pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:04.091776 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:04.091674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:04.243151 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:04.243114    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"]
Jul 24 18:25:04.245577 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:25:04.245544    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2fdceb7_8d09_4fe4_814e_69a2c8eace25.slice/crio-bf40aa6a35a756103fa5d84703990714ce10a84e7baac925a55d8c3ad162da2f WatchSource:0}: Error finding container bf40aa6a35a756103fa5d84703990714ce10a84e7baac925a55d8c3ad162da2f: Status 404 returned error can't find the container with id bf40aa6a35a756103fa5d84703990714ce10a84e7baac925a55d8c3ad162da2f
Jul 24 18:25:04.643281 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:04.643230    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs" event={"ID":"c2fdceb7-8d09-4fe4-814e-69a2c8eace25","Type":"ContainerStarted","Data":"10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390"}
Jul 24 18:25:04.643452 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:04.643299    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs" event={"ID":"c2fdceb7-8d09-4fe4-814e-69a2c8eace25","Type":"ContainerStarted","Data":"bf40aa6a35a756103fa5d84703990714ce10a84e7baac925a55d8c3ad162da2f"}
Jul 24 18:25:04.662527 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:04.662464    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs" podStartSLOduration=1.662447435 podStartE2EDuration="1.662447435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:25:04.660084194 +0000 UTC m=+2903.064025425" watchObservedRunningTime="2026-07-24 18:25:04.662447435 +0000 UTC m=+2903.066388663"
Jul 24 18:25:07.161178 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:07.161082    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:25:14.092321 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:14.092244    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:14.092321 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:14.092334    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:14.104625 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:14.104595    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:14.702824 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:14.702777    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:25:24.767303 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.767252    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"]
Jul 24 18:25:24.771176 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.771153    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:24.774642 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.774613    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"openshift-service-ca.crt\""
Jul 24 18:25:24.774809 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.774613    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"default-dockercfg-czpdh\""
Jul 24 18:25:24.774809 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.774677    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"kube-root-ca.crt\""
Jul 24 18:25:24.774809 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.774735    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c4d732\"/\"rate-test-model-c4d732-kserve-self-signed-certs\""
Jul 24 18:25:24.781220 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.781187    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"]
Jul 24 18:25:24.904510 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.904471    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-home\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:24.904510 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.904510    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-model-cache\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:24.904725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.904532    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-tmp-dir\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:24.904725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.904649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6qtn\" (UniqueName: \"kubernetes.io/projected/c30bb9af-c578-4b8b-8437-18c408829189-kube-api-access-s6qtn\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:24.904725 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.904715    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-dshm\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:24.904830 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:24.904742    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30bb9af-c578-4b8b-8437-18c408829189-tls-certs\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005442 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005396    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-home\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005442 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005444    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-model-cache\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005684 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005463    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-tmp-dir\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005684 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005492    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6qtn\" (UniqueName: \"kubernetes.io/projected/c30bb9af-c578-4b8b-8437-18c408829189-kube-api-access-s6qtn\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005684 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-dshm\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005831 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005699    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30bb9af-c578-4b8b-8437-18c408829189-tls-certs\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005831 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005809    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-home\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005937 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005828    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-tmp-dir\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.005937 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.005891    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-model-cache\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.007930 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.007907    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-dshm\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.008197 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.008181    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30bb9af-c578-4b8b-8437-18c408829189-tls-certs\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.015076 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.015051    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6qtn\" (UniqueName: \"kubernetes.io/projected/c30bb9af-c578-4b8b-8437-18c408829189-kube-api-access-s6qtn\") pod \"rate-test-model-c4d732-kserve-6475497f67-thpm7\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") " pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.082983 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.082882    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:25.223341 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.223217    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"]
Jul 24 18:25:25.225779 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:25:25.225746    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc30bb9af_c578_4b8b_8437_18c408829189.slice/crio-5eca4a09d975e6f1739f513ab29f72d47779fb0620206162d9c9eec035fa36b7 WatchSource:0}: Error finding container 5eca4a09d975e6f1739f513ab29f72d47779fb0620206162d9c9eec035fa36b7: Status 404 returned error can't find the container with id 5eca4a09d975e6f1739f513ab29f72d47779fb0620206162d9c9eec035fa36b7
Jul 24 18:25:25.744957 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.744924    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7" event={"ID":"c30bb9af-c578-4b8b-8437-18c408829189","Type":"ContainerStarted","Data":"705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9"}
Jul 24 18:25:25.745143 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.744965    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7" event={"ID":"c30bb9af-c578-4b8b-8437-18c408829189","Type":"ContainerStarted","Data":"5eca4a09d975e6f1739f513ab29f72d47779fb0620206162d9c9eec035fa36b7"}
Jul 24 18:25:25.762287 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:25.762201    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7" podStartSLOduration=1.762176849 podStartE2EDuration="1.762176849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:25:25.759902912 +0000 UTC m=+2924.163844141" watchObservedRunningTime="2026-07-24 18:25:25.762176849 +0000 UTC m=+2924.166118089"
Jul 24 18:25:28.553621 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:28.553561    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:25:35.083029 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:35.082985    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:35.083489 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:35.083046    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:35.095414 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:35.095386    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:35.808372 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:35.808334    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:25:44.460169 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:25:44.460133    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:26:06.558837 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:06.558753    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:26:10.438810 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:10.438777    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"]
Jul 24 18:26:11.435006 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:11.434972    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"]
Jul 24 18:26:23.762785 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:23.762738    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:26:25.457877 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.457835    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs" podUID="c2fdceb7-8d09-4fe4-814e-69a2c8eace25" containerName="main" containerID="cri-o://10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390" gracePeriod=45
Jul 24 18:26:25.708322 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.708226    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:26:25.785850 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.785807    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-model-cache\") pod \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") "
Jul 24 18:26:25.786057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.785899    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-dshm\") pod \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") "
Jul 24 18:26:25.786057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.785964    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tls-certs\") pod \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") "
Jul 24 18:26:25.786057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.785992    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-home\") pod \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") "
Jul 24 18:26:25.786057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786019    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpht9\" (UniqueName: \"kubernetes.io/projected/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-kube-api-access-kpht9\") pod \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") "
Jul 24 18:26:25.786057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786045    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tmp-dir\") pod \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\" (UID: \"c2fdceb7-8d09-4fe4-814e-69a2c8eace25\") "
Jul 24 18:26:25.786355 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786092    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2fdceb7-8d09-4fe4-814e-69a2c8eace25" (UID: "c2fdceb7-8d09-4fe4-814e-69a2c8eace25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:25.786355 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786208    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-home" (OuterVolumeSpecName: "home") pod "c2fdceb7-8d09-4fe4-814e-69a2c8eace25" (UID: "c2fdceb7-8d09-4fe4-814e-69a2c8eace25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:25.786461 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786424    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:25.786461 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786444    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:25.786461 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.786435    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2fdceb7-8d09-4fe4-814e-69a2c8eace25" (UID: "c2fdceb7-8d09-4fe4-814e-69a2c8eace25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:25.788182 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.788155    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2fdceb7-8d09-4fe4-814e-69a2c8eace25" (UID: "c2fdceb7-8d09-4fe4-814e-69a2c8eace25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:26:25.788182 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.788157    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-kube-api-access-kpht9" (OuterVolumeSpecName: "kube-api-access-kpht9") pod "c2fdceb7-8d09-4fe4-814e-69a2c8eace25" (UID: "c2fdceb7-8d09-4fe4-814e-69a2c8eace25"). InnerVolumeSpecName "kube-api-access-kpht9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:26:25.788367 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.788162    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-dshm" (OuterVolumeSpecName: "dshm") pod "c2fdceb7-8d09-4fe4-814e-69a2c8eace25" (UID: "c2fdceb7-8d09-4fe4-814e-69a2c8eace25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:25.887395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.887361    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:25.887395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.887389    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:25.887395 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.887400    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpht9\" (UniqueName: \"kubernetes.io/projected/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-kube-api-access-kpht9\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:25.887643 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:25.887410    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2fdceb7-8d09-4fe4-814e-69a2c8eace25-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:26.037221 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.037133    2575 generic.go:356] "Generic (PLEG): container finished" podID="c2fdceb7-8d09-4fe4-814e-69a2c8eace25" containerID="10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390" exitCode=0
Jul 24 18:26:26.037221 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.037178    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"
Jul 24 18:26:26.037221 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.037210    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs" event={"ID":"c2fdceb7-8d09-4fe4-814e-69a2c8eace25","Type":"ContainerDied","Data":"10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390"}
Jul 24 18:26:26.037522 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.037248    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs" event={"ID":"c2fdceb7-8d09-4fe4-814e-69a2c8eace25","Type":"ContainerDied","Data":"bf40aa6a35a756103fa5d84703990714ce10a84e7baac925a55d8c3ad162da2f"}
Jul 24 18:26:26.037522 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.037297    2575 scope.go:117] "RemoveContainer" containerID="10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390"
Jul 24 18:26:26.046330 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.046303    2575 scope.go:117] "RemoveContainer" containerID="10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390"
Jul 24 18:26:26.046645 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:26:26.046624    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390\": container with ID starting with 10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390 not found: ID does not exist" containerID="10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390"
Jul 24 18:26:26.046692 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.046656    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390"} err="failed to get container status \"10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390\": rpc error: code = NotFound desc = could not find container \"10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390\": container with ID starting with 10592838a1cdcfbdee7ede461a0fce26068df90e3ca0fb9078f962fed3fa0390 not found: ID does not exist"
Jul 24 18:26:26.056708 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.056673    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"]
Jul 24 18:26:26.060662 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.060633    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-100cff/rate-test-model-100cff-kserve-7488bb7bc5-l7mjs"]
Jul 24 18:26:26.138283 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.138237    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2fdceb7-8d09-4fe4-814e-69a2c8eace25" path="/var/lib/kubelet/pods/c2fdceb7-8d09-4fe4-814e-69a2c8eace25/volumes"
Jul 24 18:26:26.455363 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.455316    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7" podUID="c30bb9af-c578-4b8b-8437-18c408829189" containerName="main" containerID="cri-o://705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9" gracePeriod=45
Jul 24 18:26:26.707756 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.707686    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:26:26.794347 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794309    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6qtn\" (UniqueName: \"kubernetes.io/projected/c30bb9af-c578-4b8b-8437-18c408829189-kube-api-access-s6qtn\") pod \"c30bb9af-c578-4b8b-8437-18c408829189\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") "
Jul 24 18:26:26.794553 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794370    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30bb9af-c578-4b8b-8437-18c408829189-tls-certs\") pod \"c30bb9af-c578-4b8b-8437-18c408829189\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") "
Jul 24 18:26:26.794553 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794441    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-dshm\") pod \"c30bb9af-c578-4b8b-8437-18c408829189\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") "
Jul 24 18:26:26.794553 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794489    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-home\") pod \"c30bb9af-c578-4b8b-8437-18c408829189\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") "
Jul 24 18:26:26.794553 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794517    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-tmp-dir\") pod \"c30bb9af-c578-4b8b-8437-18c408829189\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") "
Jul 24 18:26:26.794770 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794576    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-model-cache\") pod \"c30bb9af-c578-4b8b-8437-18c408829189\" (UID: \"c30bb9af-c578-4b8b-8437-18c408829189\") "
Jul 24 18:26:26.794822 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794765    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-home" (OuterVolumeSpecName: "home") pod "c30bb9af-c578-4b8b-8437-18c408829189" (UID: "c30bb9af-c578-4b8b-8437-18c408829189"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:26.794878 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794818    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c30bb9af-c578-4b8b-8437-18c408829189" (UID: "c30bb9af-c578-4b8b-8437-18c408829189"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:26.794927 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794891    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:26.794971 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.794920    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-model-cache" (OuterVolumeSpecName: "model-cache") pod "c30bb9af-c578-4b8b-8437-18c408829189" (UID: "c30bb9af-c578-4b8b-8437-18c408829189"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:26.796637 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.796613    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-dshm" (OuterVolumeSpecName: "dshm") pod "c30bb9af-c578-4b8b-8437-18c408829189" (UID: "c30bb9af-c578-4b8b-8437-18c408829189"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:26:26.796637 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.796629    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30bb9af-c578-4b8b-8437-18c408829189-kube-api-access-s6qtn" (OuterVolumeSpecName: "kube-api-access-s6qtn") pod "c30bb9af-c578-4b8b-8437-18c408829189" (UID: "c30bb9af-c578-4b8b-8437-18c408829189"). InnerVolumeSpecName "kube-api-access-s6qtn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:26:26.796773 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.796645    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30bb9af-c578-4b8b-8437-18c408829189-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c30bb9af-c578-4b8b-8437-18c408829189" (UID: "c30bb9af-c578-4b8b-8437-18c408829189"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:26:26.896132 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.896090    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:26.896132 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.896122    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:26.896132 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.896133    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30bb9af-c578-4b8b-8437-18c408829189-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:26.896132 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.896141    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6qtn\" (UniqueName: \"kubernetes.io/projected/c30bb9af-c578-4b8b-8437-18c408829189-kube-api-access-s6qtn\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:26.896465 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:26.896152    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30bb9af-c578-4b8b-8437-18c408829189-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:27.043669 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.043570    2575 generic.go:356] "Generic (PLEG): container finished" podID="c30bb9af-c578-4b8b-8437-18c408829189" containerID="705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9" exitCode=0
Jul 24 18:26:27.043669 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.043612    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"
Jul 24 18:26:27.043669 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.043661    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7" event={"ID":"c30bb9af-c578-4b8b-8437-18c408829189","Type":"ContainerDied","Data":"705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9"}
Jul 24 18:26:27.043966 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.043699    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7" event={"ID":"c30bb9af-c578-4b8b-8437-18c408829189","Type":"ContainerDied","Data":"5eca4a09d975e6f1739f513ab29f72d47779fb0620206162d9c9eec035fa36b7"}
Jul 24 18:26:27.043966 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.043717    2575 scope.go:117] "RemoveContainer" containerID="705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9"
Jul 24 18:26:27.059095 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.059072    2575 scope.go:117] "RemoveContainer" containerID="705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9"
Jul 24 18:26:27.059421 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:26:27.059401    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9\": container with ID starting with 705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9 not found: ID does not exist" containerID="705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9"
Jul 24 18:26:27.059509 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.059434    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9"} err="failed to get container status \"705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9\": rpc error: code = NotFound desc = could not find container \"705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9\": container with ID starting with 705bb303673a54959d6b0d8bfd7e217c37cdc2bddc45998e39521b3b7f27cbb9 not found: ID does not exist"
Jul 24 18:26:27.070760 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.070723    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"]
Jul 24 18:26:27.074390 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.074364    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c4d732/rate-test-model-c4d732-kserve-6475497f67-thpm7"]
Jul 24 18:26:27.782161 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782119    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7586-139897408095696"]
Jul 24 18:26:27.782763 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782741    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30bb9af-c578-4b8b-8437-18c408829189" containerName="main"
Jul 24 18:26:27.782763 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782765    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30bb9af-c578-4b8b-8437-18c408829189" containerName="main"
Jul 24 18:26:27.782897 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782782    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2fdceb7-8d09-4fe4-814e-69a2c8eace25" containerName="main"
Jul 24 18:26:27.782897 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782789    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2fdceb7-8d09-4fe4-814e-69a2c8eace25" containerName="main"
Jul 24 18:26:27.782897 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782879    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30bb9af-c578-4b8b-8437-18c408829189" containerName="main"
Jul 24 18:26:27.783038 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.782899    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2fdceb7-8d09-4fe4-814e-69a2c8eace25" containerName="main"
Jul 24 18:26:27.787298 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.787246    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:27.789583 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.789545    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2q6hx\""
Jul 24 18:26:27.789997 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.789973    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7586-139897408095696"]
Jul 24 18:26:27.905836 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:27.905792    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-992jf\" (UniqueName: \"kubernetes.io/projected/4792f22f-f400-40d2-8cab-f9cb23ce0700-kube-api-access-992jf\") pod \"test-curl-7586-139897408095696\" (UID: \"4792f22f-f400-40d2-8cab-f9cb23ce0700\") " pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:28.007187 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:28.007154    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-992jf\" (UniqueName: \"kubernetes.io/projected/4792f22f-f400-40d2-8cab-f9cb23ce0700-kube-api-access-992jf\") pod \"test-curl-7586-139897408095696\" (UID: \"4792f22f-f400-40d2-8cab-f9cb23ce0700\") " pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:28.016202 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:28.016161    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-992jf\" (UniqueName: \"kubernetes.io/projected/4792f22f-f400-40d2-8cab-f9cb23ce0700-kube-api-access-992jf\") pod \"test-curl-7586-139897408095696\" (UID: \"4792f22f-f400-40d2-8cab-f9cb23ce0700\") " pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:28.097703 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:28.097610    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:28.138207 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:28.138175    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30bb9af-c578-4b8b-8437-18c408829189" path="/var/lib/kubelet/pods/c30bb9af-c578-4b8b-8437-18c408829189/volumes"
Jul 24 18:26:28.252180 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:28.252150    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7586-139897408095696"]
Jul 24 18:26:28.254682 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:26:28.254646    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4792f22f_f400_40d2_8cab_f9cb23ce0700.slice/crio-cda58949b43b0c9a9abe8275a57405ab6bb9e2a6ba9bddf3f977ed7a341ec817 WatchSource:0}: Error finding container cda58949b43b0c9a9abe8275a57405ab6bb9e2a6ba9bddf3f977ed7a341ec817: Status 404 returned error can't find the container with id cda58949b43b0c9a9abe8275a57405ab6bb9e2a6ba9bddf3f977ed7a341ec817
Jul 24 18:26:28.256012 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:28.255993    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:26:29.056872 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:29.056838    2575 generic.go:356] "Generic (PLEG): container finished" podID="4792f22f-f400-40d2-8cab-f9cb23ce0700" containerID="d7e534bdbaf19f0186c5e879abd32ef91bcc21eed7ce595a817093b82c24ccae" exitCode=0
Jul 24 18:26:29.057257 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:29.056920    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7586-139897408095696" event={"ID":"4792f22f-f400-40d2-8cab-f9cb23ce0700","Type":"ContainerDied","Data":"d7e534bdbaf19f0186c5e879abd32ef91bcc21eed7ce595a817093b82c24ccae"}
Jul 24 18:26:29.057257 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:29.056964    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7586-139897408095696" event={"ID":"4792f22f-f400-40d2-8cab-f9cb23ce0700","Type":"ContainerStarted","Data":"cda58949b43b0c9a9abe8275a57405ab6bb9e2a6ba9bddf3f977ed7a341ec817"}
Jul 24 18:26:30.194520 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.194492    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:30.233159 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.233127    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7586-139897408095696_4792f22f-f400-40d2-8cab-f9cb23ce0700/test-curl-7586-139897408095696/0.log"
Jul 24 18:26:30.264347 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.264310    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7586-139897408095696"]
Jul 24 18:26:30.270662 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.270635    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7586-139897408095696"]
Jul 24 18:26:30.329722 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.329631    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-992jf\" (UniqueName: \"kubernetes.io/projected/4792f22f-f400-40d2-8cab-f9cb23ce0700-kube-api-access-992jf\") pod \"4792f22f-f400-40d2-8cab-f9cb23ce0700\" (UID: \"4792f22f-f400-40d2-8cab-f9cb23ce0700\") "
Jul 24 18:26:30.331824 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.331794    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4792f22f-f400-40d2-8cab-f9cb23ce0700-kube-api-access-992jf" (OuterVolumeSpecName: "kube-api-access-992jf") pod "4792f22f-f400-40d2-8cab-f9cb23ce0700" (UID: "4792f22f-f400-40d2-8cab-f9cb23ce0700"). InnerVolumeSpecName "kube-api-access-992jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:26:30.431090 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:30.431054    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-992jf\" (UniqueName: \"kubernetes.io/projected/4792f22f-f400-40d2-8cab-f9cb23ce0700-kube-api-access-992jf\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:26:31.068606 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.068581    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cda58949b43b0c9a9abe8275a57405ab6bb9e2a6ba9bddf3f977ed7a341ec817"
Jul 24 18:26:31.068606 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.068600    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:31.077998 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:26:31.077963    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7586-139897408095696\" is forbidden: User \"system:node:ip-10-0-140-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-161.ec2.internal' and this object" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:31.107430 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.107397    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"]
Jul 24 18:26:31.107792 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.107779    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" containerName="test-curl-7586-139897408095696"
Jul 24 18:26:31.107848 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.107795    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" containerName="test-curl-7586-139897408095696"
Jul 24 18:26:31.107891 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.107864    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" containerName="test-curl-7586-139897408095696"
Jul 24 18:26:31.110892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.110875    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.113417 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:26:31.113381    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7586-139897408095696\" is forbidden: User \"system:node:ip-10-0-140-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-161.ec2.internal' and this object" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:31.113842 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.113826    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 18:26:31.113973 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.113943    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-7ee2e6-openshift-default-dockercfg-62v4t\""
Jul 24 18:26:31.131797 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.131767    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"]
Jul 24 18:26:31.236659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236618    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istiod-ca-cert\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.236659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236659    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-token\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236695    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc464\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-kube-api-access-nc464\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236729    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-data\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236770    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-certs\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236803    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-credential-socket\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236843    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-envoy\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.236980    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-podinfo\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.237217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.237012    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-socket\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.337913 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.337803    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc464\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-kube-api-access-nc464\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.337913 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.337843    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-data\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.337913 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.337877    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-certs\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.337913 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.337913    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-credential-socket\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338248 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.337955    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-envoy\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338248 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338110    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-podinfo\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338248 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338158    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-socket\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338248 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338202    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istiod-ca-cert\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338248 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338241    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-token\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338524 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-data\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338524 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338360    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-certs\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338524 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338398    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-credential-socket\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.338672 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338648    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-socket\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.339004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.338981    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istiod-ca-cert\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.340519 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.340494    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-envoy\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.340747 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.340731    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-podinfo\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.347245 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.347216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc464\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-kube-api-access-nc464\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.347403 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.347216    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-token\") pod \"e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") " pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.421835 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.421798    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:31.567099 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:31.566869    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"]
Jul 24 18:26:31.569248 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:26:31.569213    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22fd428b_cfee_47fe_b72a_9038b3b17f3d.slice/crio-ef7b395ecbf9af3bc63aa9b7c74ce9aa4c6476f7ad1545e96ca561f35fce3ea0 WatchSource:0}: Error finding container ef7b395ecbf9af3bc63aa9b7c74ce9aa4c6476f7ad1545e96ca561f35fce3ea0: Status 404 returned error can't find the container with id ef7b395ecbf9af3bc63aa9b7c74ce9aa4c6476f7ad1545e96ca561f35fce3ea0
Jul 24 18:26:32.075217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:32.075181    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" event={"ID":"22fd428b-cfee-47fe-b72a-9038b3b17f3d","Type":"ContainerStarted","Data":"ef7b395ecbf9af3bc63aa9b7c74ce9aa4c6476f7ad1545e96ca561f35fce3ea0"}
Jul 24 18:26:32.135871 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:26:32.135830    2575 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7586-139897408095696\" is forbidden: User \"system:node:ip-10-0-140-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-161.ec2.internal' and this object" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" pod="opendatahub/test-curl-7586-139897408095696"
Jul 24 18:26:32.137508 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:32.137479    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4792f22f-f400-40d2-8cab-f9cb23ce0700" path="/var/lib/kubelet/pods/4792f22f-f400-40d2-8cab-f9cb23ce0700/volumes"
Jul 24 18:26:42.293985 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:42.293949    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:26:42.310020 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:42.309993    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:26:57.228482 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:57.228442    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:26:57.228934 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:57.228535    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:26:57.228934 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:57.228578    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:26:58.203760 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:58.203722    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" event={"ID":"22fd428b-cfee-47fe-b72a-9038b3b17f3d","Type":"ContainerStarted","Data":"02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"}
Jul 24 18:26:58.230420 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:58.230354    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" podStartSLOduration=1.5735515599999998 podStartE2EDuration="27.230331433s" totalImagesPullingTime="25.656779873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:26:58.228922833 +0000 UTC m=+3016.632864061" watchObservedRunningTime="2026-07-24 18:26:58.230331433 +0000 UTC m=+3016.634272664"
Jul 24 18:26:58.422030 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:58.421990    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:58.427146 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:58.427117    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:59.208882 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:59.208850    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:26:59.209920 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:26:59.209899    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:27:05.661071 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:05.661033    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:27:13.286489 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:13.286450    2575 generic.go:356] "Generic (PLEG): container finished" podID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerID="02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b" exitCode=0
Jul 24 18:27:13.286938 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:13.286530    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" event={"ID":"22fd428b-cfee-47fe-b72a-9038b3b17f3d","Type":"ContainerDied","Data":"02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"}
Jul 24 18:27:13.286984 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:13.286954    2575 scope.go:117] "RemoveContainer" containerID="02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"
Jul 24 18:27:13.289117 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:13.289085    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:27:13.289238 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:13.289163    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:27:13.289238 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:13.289195    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:27:14.209422 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:14.209377    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:27:14.293082 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:14.293050    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" event={"ID":"22fd428b-cfee-47fe-b72a-9038b3b17f3d","Type":"ContainerStarted","Data":"449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06"}
Jul 24 18:27:14.422683 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:14.422654    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:27:14.423795 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:14.423763    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 24 18:27:14.423929 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:14.423816    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 24 18:27:15.422769 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:15.422716    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 24 18:27:15.423194 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:15.422797    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 24 18:27:16.423130 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:16.423085    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 24 18:27:16.423659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:16.423152    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 24 18:27:17.422909 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:17.422866    2575 patch_prober.go:28] interesting pod/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 24 18:27:17.423135 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:17.422933    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 24 18:27:18.427281 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:18.427239    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:27:18.427684 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:18.427524    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:27:18.428194 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:18.428176    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:27:19.634848 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.634810    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"]
Jul 24 18:27:19.643555 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.643534    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.649593 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.649542    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e18e67-dockercfg-jvfv9\""
Jul 24 18:27:19.649989 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.649969    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e18e67\""
Jul 24 18:27:19.652141 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.652112    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"]
Jul 24 18:27:19.677133 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.677096    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"]
Jul 24 18:27:19.680792 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.680765    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.690915 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.690887    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"]
Jul 24 18:27:19.806528 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.806491    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d97298-88bd-44f8-9713-7ebe61d53208-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.806752 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.806534    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zv8p\" (UniqueName: \"kubernetes.io/projected/e6d97298-88bd-44f8-9713-7ebe61d53208-kube-api-access-4zv8p\") pod \"payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.806752 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.806578    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np66p\" (UniqueName: \"kubernetes.io/projected/d0b90007-c72c-4883-a211-2854050de426-kube-api-access-np66p\") pod \"payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.806752 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.806649    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b90007-c72c-4883-a211-2854050de426-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.908236 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.908127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b90007-c72c-4883-a211-2854050de426-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.908236 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.908211    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d97298-88bd-44f8-9713-7ebe61d53208-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.908600 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.908367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zv8p\" (UniqueName: \"kubernetes.io/projected/e6d97298-88bd-44f8-9713-7ebe61d53208-kube-api-access-4zv8p\") pod \"payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.908600 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.908449    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np66p\" (UniqueName: \"kubernetes.io/projected/d0b90007-c72c-4883-a211-2854050de426-kube-api-access-np66p\") pod \"payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.909049 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.909023    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b90007-c72c-4883-a211-2854050de426-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.909142 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.909125    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d97298-88bd-44f8-9713-7ebe61d53208-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.923008 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.922982    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np66p\" (UniqueName: \"kubernetes.io/projected/d0b90007-c72c-4883-a211-2854050de426-kube-api-access-np66p\") pod \"payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:19.923142 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.923099    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zv8p\" (UniqueName: \"kubernetes.io/projected/e6d97298-88bd-44f8-9713-7ebe61d53208-kube-api-access-4zv8p\") pod \"payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.955084 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.955050    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:19.993330 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:19.993296    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:20.115409 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:20.115379    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"]
Jul 24 18:27:20.116850 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:27:20.116821    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d97298_88bd_44f8_9713_7ebe61d53208.slice/crio-382cd084e5e93a140395a5ee5447b8b4b191fa68a7eefad42c44996957994c81 WatchSource:0}: Error finding container 382cd084e5e93a140395a5ee5447b8b4b191fa68a7eefad42c44996957994c81: Status 404 returned error can't find the container with id 382cd084e5e93a140395a5ee5447b8b4b191fa68a7eefad42c44996957994c81
Jul 24 18:27:20.149180 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:20.149151    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"]
Jul 24 18:27:20.151379 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:27:20.151347    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0b90007_c72c_4883_a211_2854050de426.slice/crio-b6a230fe9d4c4b53dce9987ca0cb6b144dc9bb0d8704a093dbb6587479fffafc WatchSource:0}: Error finding container b6a230fe9d4c4b53dce9987ca0cb6b144dc9bb0d8704a093dbb6587479fffafc: Status 404 returned error can't find the container with id b6a230fe9d4c4b53dce9987ca0cb6b144dc9bb0d8704a093dbb6587479fffafc
Jul 24 18:27:20.324428 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:20.324382    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9" event={"ID":"d0b90007-c72c-4883-a211-2854050de426","Type":"ContainerStarted","Data":"b6a230fe9d4c4b53dce9987ca0cb6b144dc9bb0d8704a093dbb6587479fffafc"}
Jul 24 18:27:20.325561 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:20.325541    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5" event={"ID":"e6d97298-88bd-44f8-9713-7ebe61d53208","Type":"ContainerStarted","Data":"382cd084e5e93a140395a5ee5447b8b4b191fa68a7eefad42c44996957994c81"}
Jul 24 18:27:23.345911 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:23.345857    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9" event={"ID":"d0b90007-c72c-4883-a211-2854050de426","Type":"ContainerStarted","Data":"0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80"}
Jul 24 18:27:23.346496 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:23.345963    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:23.347577 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:23.347548    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5" event={"ID":"e6d97298-88bd-44f8-9713-7ebe61d53208","Type":"ContainerStarted","Data":"fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b"}
Jul 24 18:27:23.347720 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:23.347659    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:23.364488 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:23.364422    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9" podStartSLOduration=1.828335796 podStartE2EDuration="4.364402256s" totalImagesPullingTime="2.53606646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:23.36265273 +0000 UTC m=+3041.766593979" watchObservedRunningTime="2026-07-24 18:27:23.364402256 +0000 UTC m=+3041.768343476"
Jul 24 18:27:25.255587 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:25.255519    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5" podStartSLOduration=3.678912421 podStartE2EDuration="6.255493957s" totalImagesPullingTime="2.576581536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:23.37712921 +0000 UTC m=+3041.781070430" watchObservedRunningTime="2026-07-24 18:27:25.255493957 +0000 UTC m=+3043.659435189"
Jul 24 18:27:25.257222 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:25.257191    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:27:34.354873 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:34.354788    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:27:34.355249 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:34.355159    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:27:45.999582 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:45.999548    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9_d0b90007-c72c-4883-a211-2854050de426/payload-processing/0.log"
Jul 24 18:27:50.983386 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.983346    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"]
Jul 24 18:27:50.987062 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.987041    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:50.990581 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.990554    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7ee2e6\"/\"kube-root-ca.crt\""
Jul 24 18:27:50.991390 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.991366    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7ee2e6\"/\"default-dockercfg-gj8r6\""
Jul 24 18:27:50.991512 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.991367    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7ee2e6\"/\"openshift-service-ca.crt\""
Jul 24 18:27:50.991581 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.991528    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7ee2e6\"/\"ipp-route-7ee2e6-kserve-self-signed-certs\""
Jul 24 18:27:50.999953 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:50.999923    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"]
Jul 24 18:27:51.097844 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.097806    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa567d8-e56a-493f-b028-cf3328b68452-tls-certs\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.098045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.097856    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx42q\" (UniqueName: \"kubernetes.io/projected/3aa567d8-e56a-493f-b028-cf3328b68452-kube-api-access-sx42q\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.098045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.097886    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-model-cache\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.098045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.097956    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-home\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.098045 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.097998    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-tmp-dir\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.098256 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.098117    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-dshm\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.199614 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.199575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx42q\" (UniqueName: \"kubernetes.io/projected/3aa567d8-e56a-493f-b028-cf3328b68452-kube-api-access-sx42q\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.199614 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.199624    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-model-cache\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.199863 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.199665    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-home\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.199863 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.199695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-tmp-dir\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.199863 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.199745    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-dshm\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.199863 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.199785    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa567d8-e56a-493f-b028-cf3328b68452-tls-certs\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.200167 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.200143    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-model-cache\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.200259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.200160    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-home\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.200259 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.200211    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-tmp-dir\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.202066 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.202043    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-dshm\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.202344 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.202327    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa567d8-e56a-493f-b028-cf3328b68452-tls-certs\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.207839 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.207809    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx42q\" (UniqueName: \"kubernetes.io/projected/3aa567d8-e56a-493f-b028-cf3328b68452-kube-api-access-sx42q\") pod \"ipp-route-7ee2e6-kserve-57d78784b7-qp9gr\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") " pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.296981 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.296881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:27:51.445078 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.445049    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"]
Jul 24 18:27:51.447319 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:27:51.447290    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aa567d8_e56a_493f_b028_cf3328b68452.slice/crio-e15508daa730ce2d683b9e5aabe17318056a9108e12177dd1e19a7b1e24d4b01 WatchSource:0}: Error finding container e15508daa730ce2d683b9e5aabe17318056a9108e12177dd1e19a7b1e24d4b01: Status 404 returned error can't find the container with id e15508daa730ce2d683b9e5aabe17318056a9108e12177dd1e19a7b1e24d4b01
Jul 24 18:27:51.487616 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:51.487585    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr" event={"ID":"3aa567d8-e56a-493f-b028-cf3328b68452","Type":"ContainerStarted","Data":"e15508daa730ce2d683b9e5aabe17318056a9108e12177dd1e19a7b1e24d4b01"}
Jul 24 18:27:52.494095 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:52.494053    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr" event={"ID":"3aa567d8-e56a-493f-b028-cf3328b68452","Type":"ContainerStarted","Data":"3c6b590f8c771abdbe2c766e69bfdd61e5992e58c9bd9c611128476de7e09794"}
Jul 24 18:27:52.514328 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:52.514259    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr" podStartSLOduration=2.514240585 podStartE2EDuration="2.514240585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:27:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:27:52.512948536 +0000 UTC m=+3070.916889765" watchObservedRunningTime="2026-07-24 18:27:52.514240585 +0000 UTC m=+3070.918181814"
Jul 24 18:27:55.267956 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:27:55.267912    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:28:01.298051 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:01.297998    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:28:01.298545 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:01.298079    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:28:01.310866 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:01.310836    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:28:01.548607 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:01.548516    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:28:11.166062 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:11.166025    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:28:22.870116 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:22.870079    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9_d0b90007-c72c-4883-a211-2854050de426/payload-processing/0.log"
Jul 24 18:28:23.449763 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.449718    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"]
Jul 24 18:28:23.453580 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.453552    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.456015 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.455977    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-a2a4d8-openshift-default-dockercfg-ldzbv\""
Jul 24 18:28:23.465629 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.465596    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"]
Jul 24 18:28:23.512753 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.512716    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-certs\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.512977 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.512768    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-socket\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.512977 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.512840    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb12923a-df24-43fb-81e1-28614f73ef25-istiod-ca-cert\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.512977 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.512924    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb12923a-df24-43fb-81e1-28614f73ef25-istio-podinfo\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.512977 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.512971    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-data\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.513212 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.513010    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-istio-token\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.513212 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.513066    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rzmw\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-kube-api-access-6rzmw\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.513212 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.513119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-credential-socket\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.513212 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.513156    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-envoy\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614615 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614573    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-socket\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614615 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb12923a-df24-43fb-81e1-28614f73ef25-istiod-ca-cert\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614645    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb12923a-df24-43fb-81e1-28614f73ef25-istio-podinfo\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614663    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-data\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614690    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-istio-token\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614723    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rzmw\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-kube-api-access-6rzmw\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614748    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-credential-socket\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614783    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-envoy\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.614892 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.614837    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-certs\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.615247 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.615019    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-socket\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.615247 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.615187    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-data\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.615247 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.615229    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-credential-socket\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.615431 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.615337    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-certs\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.615633 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.615613    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb12923a-df24-43fb-81e1-28614f73ef25-istiod-ca-cert\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.617247 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.617223    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-envoy\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.617397 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.617379    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb12923a-df24-43fb-81e1-28614f73ef25-istio-podinfo\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.624331 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.624252    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-istio-token\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.624435 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.624394    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rzmw\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-kube-api-access-6rzmw\") pod \"e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") " pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.766839 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.766734    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:23.920423 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.920390    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"]
Jul 24 18:28:23.922814 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:28:23.922781    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb12923a_df24_43fb_81e1_28614f73ef25.slice/crio-9194a8ff7d69b1d69e2bf7cb9670f28f5db65e13f9c78029c07e15e48f84998e WatchSource:0}: Error finding container 9194a8ff7d69b1d69e2bf7cb9670f28f5db65e13f9c78029c07e15e48f84998e: Status 404 returned error can't find the container with id 9194a8ff7d69b1d69e2bf7cb9670f28f5db65e13f9c78029c07e15e48f84998e
Jul 24 18:28:23.925115 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.925080    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:28:23.925217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.925167    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:28:23.925217 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:23.925203    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:28:24.652569 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:24.652529    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" event={"ID":"eb12923a-df24-43fb-81e1-28614f73ef25","Type":"ContainerStarted","Data":"06c981b3594785b057691898cdea36e5e936ac60f2603816f11c51ed48c31c21"}
Jul 24 18:28:24.652780 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:24.652594    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" event={"ID":"eb12923a-df24-43fb-81e1-28614f73ef25","Type":"ContainerStarted","Data":"9194a8ff7d69b1d69e2bf7cb9670f28f5db65e13f9c78029c07e15e48f84998e"}
Jul 24 18:28:24.674603 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:24.674541    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podStartSLOduration=1.67452214 podStartE2EDuration="1.67452214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:28:24.671696626 +0000 UTC m=+3103.075637868" watchObservedRunningTime="2026-07-24 18:28:24.67452214 +0000 UTC m=+3103.078463428"
Jul 24 18:28:24.767461 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:24.767427    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:24.773433 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:24.773355    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:25.658050 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:25.658018    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:25.659041 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:25.659024    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:27.538879 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.538835    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"]
Jul 24 18:28:27.542914 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.542885    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.546502 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.546466    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-a2a4d8\""
Jul 24 18:28:27.546658 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.546539    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-a2a4d8-dockercfg-46mn5\""
Jul 24 18:28:27.560364 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.560328    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"]
Jul 24 18:28:27.654881 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.654843    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84spb\" (UniqueName: \"kubernetes.io/projected/ee521783-2c52-482f-bc92-3fbbd9fe56d4-kube-api-access-84spb\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.655161 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.655017    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee521783-2c52-482f-bc92-3fbbd9fe56d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.756757 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.756712    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee521783-2c52-482f-bc92-3fbbd9fe56d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.756995 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.756822    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84spb\" (UniqueName: \"kubernetes.io/projected/ee521783-2c52-482f-bc92-3fbbd9fe56d4-kube-api-access-84spb\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.757649 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.757628    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee521783-2c52-482f-bc92-3fbbd9fe56d4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.767064 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.767020    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84spb\" (UniqueName: \"kubernetes.io/projected/ee521783-2c52-482f-bc92-3fbbd9fe56d4-kube-api-access-84spb\") pod \"payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.854104 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.854004    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:27.997496 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:27.997462    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"]
Jul 24 18:28:27.999396 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:28:27.999369    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee521783_2c52_482f_bc92_3fbbd9fe56d4.slice/crio-5698dd5d3ea73b0ed05179f5d1e3d7b8c132619a1169c742f2eb2c55ee4eabc1 WatchSource:0}: Error finding container 5698dd5d3ea73b0ed05179f5d1e3d7b8c132619a1169c742f2eb2c55ee4eabc1: Status 404 returned error can't find the container with id 5698dd5d3ea73b0ed05179f5d1e3d7b8c132619a1169c742f2eb2c55ee4eabc1
Jul 24 18:28:28.674021 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:28.673978    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp" event={"ID":"ee521783-2c52-482f-bc92-3fbbd9fe56d4","Type":"ContainerStarted","Data":"9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7"}
Jul 24 18:28:28.674460 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:28.674039    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp" event={"ID":"ee521783-2c52-482f-bc92-3fbbd9fe56d4","Type":"ContainerStarted","Data":"5698dd5d3ea73b0ed05179f5d1e3d7b8c132619a1169c742f2eb2c55ee4eabc1"}
Jul 24 18:28:28.674460 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:28.674150    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:28.693678 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:28.693616    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp" podStartSLOduration=1.693595331 podStartE2EDuration="1.693595331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:28:28.69134849 +0000 UTC m=+3107.095289720" watchObservedRunningTime="2026-07-24 18:28:28.693595331 +0000 UTC m=+3107.097536606"
Jul 24 18:28:30.987187 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:30.987145    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:28:39.681906 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.681873    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:39.737240 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.737206    2575 generic.go:356] "Generic (PLEG): container finished" podID="eb12923a-df24-43fb-81e1-28614f73ef25" containerID="06c981b3594785b057691898cdea36e5e936ac60f2603816f11c51ed48c31c21" exitCode=0
Jul 24 18:28:39.737453 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.737297    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" event={"ID":"eb12923a-df24-43fb-81e1-28614f73ef25","Type":"ContainerDied","Data":"06c981b3594785b057691898cdea36e5e936ac60f2603816f11c51ed48c31c21"}
Jul 24 18:28:39.737794 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.737777    2575 scope.go:117] "RemoveContainer" containerID="06c981b3594785b057691898cdea36e5e936ac60f2603816f11c51ed48c31c21"
Jul 24 18:28:39.739049 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.739019    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:28:39.739138 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.739089    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:28:39.739138 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:39.739123    2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 24 18:28:40.658347 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:40.658304    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:40.745676 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:40.745641    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" event={"ID":"eb12923a-df24-43fb-81e1-28614f73ef25","Type":"ContainerStarted","Data":"55a06a7fdeb3cb6191c073315bf10e8c098468d51ffee80fa350fd316529d2be"}
Jul 24 18:28:40.766944 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:40.766898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:40.768920 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:40.768867    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 18:28:40.769237 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:40.769190    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 18:28:41.767923 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:41.767875    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 18:28:41.768393 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:41.767954    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 18:28:42.767470 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:42.767426    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 18:28:42.767667 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:42.767502    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 18:28:43.767371 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:43.767334    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 18:28:43.767800 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:43.767404    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 18:28:44.767313 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:44.767275    2575 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 24 18:28:44.767491 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:44.767343    2575 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 24 18:28:45.771537 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:45.771503    2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:45.771997 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:45.771898    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:45.772797 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:45.772775    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:28:52.750192 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:52.750152    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"]
Jul 24 18:28:52.750916 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:52.750386    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp" podUID="ee521783-2c52-482f-bc92-3fbbd9fe56d4" containerName="payload-pre-processing" containerID="cri-o://9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7" gracePeriod=30
Jul 24 18:28:53.017704 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.017667    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:53.020734 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.020701    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84spb\" (UniqueName: \"kubernetes.io/projected/ee521783-2c52-482f-bc92-3fbbd9fe56d4-kube-api-access-84spb\") pod \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") "
Jul 24 18:28:53.020868 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.020767    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee521783-2c52-482f-bc92-3fbbd9fe56d4-plugins-config-volume\") pod \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\" (UID: \"ee521783-2c52-482f-bc92-3fbbd9fe56d4\") "
Jul 24 18:28:53.021226 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.021191    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee521783-2c52-482f-bc92-3fbbd9fe56d4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ee521783-2c52-482f-bc92-3fbbd9fe56d4" (UID: "ee521783-2c52-482f-bc92-3fbbd9fe56d4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:28:53.023134 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.023098    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee521783-2c52-482f-bc92-3fbbd9fe56d4-kube-api-access-84spb" (OuterVolumeSpecName: "kube-api-access-84spb") pod "ee521783-2c52-482f-bc92-3fbbd9fe56d4" (UID: "ee521783-2c52-482f-bc92-3fbbd9fe56d4"). InnerVolumeSpecName "kube-api-access-84spb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:28:53.121486 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.121449    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee521783-2c52-482f-bc92-3fbbd9fe56d4-plugins-config-volume\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:28:53.121486 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.121479    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84spb\" (UniqueName: \"kubernetes.io/projected/ee521783-2c52-482f-bc92-3fbbd9fe56d4-kube-api-access-84spb\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:28:53.818416 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.818378    2575 generic.go:356] "Generic (PLEG): container finished" podID="ee521783-2c52-482f-bc92-3fbbd9fe56d4" containerID="9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7" exitCode=0
Jul 24 18:28:53.819004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.818424    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"
Jul 24 18:28:53.819004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.818450    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp" event={"ID":"ee521783-2c52-482f-bc92-3fbbd9fe56d4","Type":"ContainerDied","Data":"9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7"}
Jul 24 18:28:53.819004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.818502    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp" event={"ID":"ee521783-2c52-482f-bc92-3fbbd9fe56d4","Type":"ContainerDied","Data":"5698dd5d3ea73b0ed05179f5d1e3d7b8c132619a1169c742f2eb2c55ee4eabc1"}
Jul 24 18:28:53.819004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.818534    2575 scope.go:117] "RemoveContainer" containerID="9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7"
Jul 24 18:28:53.827816 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.827792    2575 scope.go:117] "RemoveContainer" containerID="9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7"
Jul 24 18:28:53.828099 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:28:53.828075    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7\": container with ID starting with 9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7 not found: ID does not exist" containerID="9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7"
Jul 24 18:28:53.828158 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.828109    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7"} err="failed to get container status \"9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7\": rpc error: code = NotFound desc = could not find container \"9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7\": container with ID starting with 9c0acb5f4ce08f5866f328cdbd5d0075986c27f787e082a8e157ae30922dd1f7 not found: ID does not exist"
Jul 24 18:28:53.841105 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.841065    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"]
Jul 24 18:28:53.847067 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:53.847040    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a2a4d8-7854c9d868-jrjbp"]
Jul 24 18:28:54.137392 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:54.137313    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee521783-2c52-482f-bc92-3fbbd9fe56d4" path="/var/lib/kubelet/pods/ee521783-2c52-482f-bc92-3fbbd9fe56d4/volumes"
Jul 24 18:28:57.366081 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:28:57.366041    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:29:03.772004 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:03.771915    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"]
Jul 24 18:29:03.772461 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:03.772159    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy" containerID="cri-o://55a06a7fdeb3cb6191c073315bf10e8c098468d51ffee80fa350fd316529d2be" gracePeriod=30
Jul 24 18:29:08.896676 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:08.896635    2575 generic.go:356] "Generic (PLEG): container finished" podID="eb12923a-df24-43fb-81e1-28614f73ef25" containerID="55a06a7fdeb3cb6191c073315bf10e8c098468d51ffee80fa350fd316529d2be" exitCode=0
Jul 24 18:29:08.897117 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:08.896704    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" event={"ID":"eb12923a-df24-43fb-81e1-28614f73ef25","Type":"ContainerDied","Data":"55a06a7fdeb3cb6191c073315bf10e8c098468d51ffee80fa350fd316529d2be"}
Jul 24 18:29:08.897117 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:08.896769    2575 scope.go:117] "RemoveContainer" containerID="06c981b3594785b057691898cdea36e5e936ac60f2603816f11c51ed48c31c21"
Jul 24 18:29:09.036016 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.035941    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:29:09.084087 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084057    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-credential-socket\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084102    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb12923a-df24-43fb-81e1-28614f73ef25-istiod-ca-cert\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084172    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-certs\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084200    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rzmw\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-kube-api-access-6rzmw\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084246    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-envoy\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084530 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084421    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-socket\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084530 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084459    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:09.084640 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084554    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:09.084640 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084579    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-data\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084749 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084652    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-istio-token\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084749 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084677    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb12923a-df24-43fb-81e1-28614f73ef25-istio-podinfo\") pod \"eb12923a-df24-43fb-81e1-28614f73ef25\" (UID: \"eb12923a-df24-43fb-81e1-28614f73ef25\") "
Jul 24 18:29:09.084916 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.084872    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb12923a-df24-43fb-81e1-28614f73ef25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:29:09.085154 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.085083    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-credential-socket\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.085154 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.085104    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb12923a-df24-43fb-81e1-28614f73ef25-istiod-ca-cert\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.085154 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.085118    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.085436 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.085400    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:09.085535 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.085514    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-data" (OuterVolumeSpecName: "istio-data") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:09.087107 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.087083    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb12923a-df24-43fb-81e1-28614f73ef25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:29:09.087222 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.087115    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-kube-api-access-6rzmw" (OuterVolumeSpecName: "kube-api-access-6rzmw") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "kube-api-access-6rzmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:29:09.087315 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.087286    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:09.087315 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.087302    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-istio-token" (OuterVolumeSpecName: "istio-token") pod "eb12923a-df24-43fb-81e1-28614f73ef25" (UID: "eb12923a-df24-43fb-81e1-28614f73ef25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:29:09.185673 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.185631    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-data\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.185673 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.185663    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-istio-token\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.185673 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.185672    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb12923a-df24-43fb-81e1-28614f73ef25-istio-podinfo\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.185673 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.185681    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rzmw\" (UniqueName: \"kubernetes.io/projected/eb12923a-df24-43fb-81e1-28614f73ef25-kube-api-access-6rzmw\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.185961 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.185691    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-istio-envoy\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.185961 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.185699    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb12923a-df24-43fb-81e1-28614f73ef25-workload-socket\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:09.904201 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.904161    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl" event={"ID":"eb12923a-df24-43fb-81e1-28614f73ef25","Type":"ContainerDied","Data":"9194a8ff7d69b1d69e2bf7cb9670f28f5db65e13f9c78029c07e15e48f84998e"}
Jul 24 18:29:09.904742 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.904225    2575 scope.go:117] "RemoveContainer" containerID="55a06a7fdeb3cb6191c073315bf10e8c098468d51ffee80fa350fd316529d2be"
Jul 24 18:29:09.904742 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.904174    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"
Jul 24 18:29:09.928733 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.928696    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"]
Jul 24 18:29:09.932634 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:09.932601    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a2a4d8-openshift-default-fd9bb7c68-sh7pl"]
Jul 24 18:29:10.139499 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:10.139466    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" path="/var/lib/kubelet/pods/eb12923a-df24-43fb-81e1-28614f73ef25/volumes"
Jul 24 18:29:21.964919 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:21.964877    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:29:32.662330 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:32.662287    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:29:32.847131 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:32.847093    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"]
Jul 24 18:29:47.867089 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:47.867030    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr" podUID="3aa567d8-e56a-493f-b028-cf3328b68452" containerName="main" containerID="cri-o://3c6b590f8c771abdbe2c766e69bfdd61e5992e58c9bd9c611128476de7e09794" gracePeriod=45
Jul 24 18:29:48.116087 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.116048    2575 generic.go:356] "Generic (PLEG): container finished" podID="3aa567d8-e56a-493f-b028-cf3328b68452" containerID="3c6b590f8c771abdbe2c766e69bfdd61e5992e58c9bd9c611128476de7e09794" exitCode=0
Jul 24 18:29:48.116301 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.116082    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr" event={"ID":"3aa567d8-e56a-493f-b028-cf3328b68452","Type":"ContainerDied","Data":"3c6b590f8c771abdbe2c766e69bfdd61e5992e58c9bd9c611128476de7e09794"}
Jul 24 18:29:48.116301 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.116127    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr" event={"ID":"3aa567d8-e56a-493f-b028-cf3328b68452","Type":"ContainerDied","Data":"e15508daa730ce2d683b9e5aabe17318056a9108e12177dd1e19a7b1e24d4b01"}
Jul 24 18:29:48.116301 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.116138    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e15508daa730ce2d683b9e5aabe17318056a9108e12177dd1e19a7b1e24d4b01"
Jul 24 18:29:48.124127 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.124052    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:29:48.262494 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262455    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx42q\" (UniqueName: \"kubernetes.io/projected/3aa567d8-e56a-493f-b028-cf3328b68452-kube-api-access-sx42q\") pod \"3aa567d8-e56a-493f-b028-cf3328b68452\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") "
Jul 24 18:29:48.262679 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262518    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-model-cache\") pod \"3aa567d8-e56a-493f-b028-cf3328b68452\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") "
Jul 24 18:29:48.262679 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262565    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-home\") pod \"3aa567d8-e56a-493f-b028-cf3328b68452\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") "
Jul 24 18:29:48.262679 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262631    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-dshm\") pod \"3aa567d8-e56a-493f-b028-cf3328b68452\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") "
Jul 24 18:29:48.262848 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262706    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-tmp-dir\") pod \"3aa567d8-e56a-493f-b028-cf3328b68452\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") "
Jul 24 18:29:48.262848 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262738    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa567d8-e56a-493f-b028-cf3328b68452-tls-certs\") pod \"3aa567d8-e56a-493f-b028-cf3328b68452\" (UID: \"3aa567d8-e56a-493f-b028-cf3328b68452\") "
Jul 24 18:29:48.262848 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262792    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-model-cache" (OuterVolumeSpecName: "model-cache") pod "3aa567d8-e56a-493f-b028-cf3328b68452" (UID: "3aa567d8-e56a-493f-b028-cf3328b68452"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:48.262848 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.262837    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-home" (OuterVolumeSpecName: "home") pod "3aa567d8-e56a-493f-b028-cf3328b68452" (UID: "3aa567d8-e56a-493f-b028-cf3328b68452"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:48.263063 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.263028    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3aa567d8-e56a-493f-b028-cf3328b68452" (UID: "3aa567d8-e56a-493f-b028-cf3328b68452"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:48.263383 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.263360    2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-tmp-dir\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:48.263566 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.263541    2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-model-cache\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:48.263566 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.263566    2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-home\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:48.265035 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.265009    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-dshm" (OuterVolumeSpecName: "dshm") pod "3aa567d8-e56a-493f-b028-cf3328b68452" (UID: "3aa567d8-e56a-493f-b028-cf3328b68452"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:29:48.265164 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.265044    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3aa567d8-e56a-493f-b028-cf3328b68452-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3aa567d8-e56a-493f-b028-cf3328b68452" (UID: "3aa567d8-e56a-493f-b028-cf3328b68452"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 18:29:48.265164 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.265149    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aa567d8-e56a-493f-b028-cf3328b68452-kube-api-access-sx42q" (OuterVolumeSpecName: "kube-api-access-sx42q") pod "3aa567d8-e56a-493f-b028-cf3328b68452" (UID: "3aa567d8-e56a-493f-b028-cf3328b68452"). InnerVolumeSpecName "kube-api-access-sx42q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:29:48.364694 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.364638    2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aa567d8-e56a-493f-b028-cf3328b68452-dshm\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:48.364694 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.364685    2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aa567d8-e56a-493f-b028-cf3328b68452-tls-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:48.364694 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:48.364698    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx42q\" (UniqueName: \"kubernetes.io/projected/3aa567d8-e56a-493f-b028-cf3328b68452-kube-api-access-sx42q\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:29:49.121849 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:49.121805    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"
Jul 24 18:29:49.143415 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:49.143378    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"]
Jul 24 18:29:49.147465 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:49.147430    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-7ee2e6/ipp-route-7ee2e6-kserve-57d78784b7-qp9gr"]
Jul 24 18:29:50.138543 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:29:50.138506    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aa567d8-e56a-493f-b028-cf3328b68452" path="/var/lib/kubelet/pods/3aa567d8-e56a-493f-b028-cf3328b68452/volumes"
Jul 24 18:30:21.760043 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:21.759999    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"]
Jul 24 18:30:21.760591 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:21.760321    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy" containerID="cri-o://449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06" gracePeriod=30
Jul 24 18:30:27.025942 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.025861    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:30:27.155111 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155067    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istiod-ca-cert\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155111 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155119    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc464\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-kube-api-access-nc464\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155424 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155163    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-credential-socket\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155424 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155218    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-envoy\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155424 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155323    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-podinfo\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155424 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155403    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-socket\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155640 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155433    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-token\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155640 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155461    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-certs\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155640 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155509    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-data\") pod \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\" (UID: \"22fd428b-cfee-47fe-b72a-9038b3b17f3d\") "
Jul 24 18:30:27.155805 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155671    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:30:27.155805 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155584    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:30:27.155898 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155868    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:30:27.155960 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155911    2575 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istiod-ca-cert\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.155960 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.155931    2575 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-socket\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.156347 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.156117    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:30:27.156679 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.156650    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-data" (OuterVolumeSpecName: "istio-data") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:30:27.157791 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.157759    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 18:30:27.157901 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.157825    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 18:30:27.157901 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.157825    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-kube-api-access-nc464" (OuterVolumeSpecName: "kube-api-access-nc464") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "kube-api-access-nc464". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:30:27.158144 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.158125    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-token" (OuterVolumeSpecName: "istio-token") pod "22fd428b-cfee-47fe-b72a-9038b3b17f3d" (UID: "22fd428b-cfee-47fe-b72a-9038b3b17f3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:30:27.257483 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257429    2575 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-envoy\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.257483 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257476    2575 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-podinfo\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.257483 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257492    2575 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-token\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.257483 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257504    2575 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-workload-certs\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.257824 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257516    2575 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-istio-data\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.257824 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257529    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc464\" (UniqueName: \"kubernetes.io/projected/22fd428b-cfee-47fe-b72a-9038b3b17f3d-kube-api-access-nc464\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.257824 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.257543    2575 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22fd428b-cfee-47fe-b72a-9038b3b17f3d-credential-socket\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:27.322699 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.322592    2575 generic.go:356] "Generic (PLEG): container finished" podID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerID="449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06" exitCode=0
Jul 24 18:30:27.322699 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.322648    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"
Jul 24 18:30:27.322699 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.322674    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" event={"ID":"22fd428b-cfee-47fe-b72a-9038b3b17f3d","Type":"ContainerDied","Data":"449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06"}
Jul 24 18:30:27.322988 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.322756    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k" event={"ID":"22fd428b-cfee-47fe-b72a-9038b3b17f3d","Type":"ContainerDied","Data":"ef7b395ecbf9af3bc63aa9b7c74ce9aa4c6476f7ad1545e96ca561f35fce3ea0"}
Jul 24 18:30:27.322988 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.322789    2575 scope.go:117] "RemoveContainer" containerID="449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06"
Jul 24 18:30:27.332820 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.332795    2575 scope.go:117] "RemoveContainer" containerID="02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"
Jul 24 18:30:27.342193 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.342093    2575 scope.go:117] "RemoveContainer" containerID="449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06"
Jul 24 18:30:27.342573 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:30:27.342513    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06\": container with ID starting with 449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06 not found: ID does not exist" containerID="449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06"
Jul 24 18:30:27.342573 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.342559    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06"} err="failed to get container status \"449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06\": rpc error: code = NotFound desc = could not find container \"449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06\": container with ID starting with 449cdb4b93374827ba0271df603304906b6ef250786753adb1f580c52afe7e06 not found: ID does not exist"
Jul 24 18:30:27.342759 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.342587    2575 scope.go:117] "RemoveContainer" containerID="02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"
Jul 24 18:30:27.342939 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:30:27.342920    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b\": container with ID starting with 02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b not found: ID does not exist" containerID="02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"
Jul 24 18:30:27.342995 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.342944    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b"} err="failed to get container status \"02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b\": rpc error: code = NotFound desc = could not find container \"02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b\": container with ID starting with 02f2a0120ea1fad2b1079ee23a6c5b56a52b0a5b1a5deed0fa38a17007a4b83b not found: ID does not exist"
Jul 24 18:30:27.344144 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.344121    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"]
Jul 24 18:30:27.350757 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:27.350716    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-7ee2e6-openshift-default-56c8b6b774-px29k"]
Jul 24 18:30:28.138130 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:28.138096    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" path="/var/lib/kubelet/pods/22fd428b-cfee-47fe-b72a-9038b3b17f3d/volumes"
Jul 24 18:30:32.803812 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:32.803773    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"]
Jul 24 18:30:32.804333 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:32.804056    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9" podUID="d0b90007-c72c-4883-a211-2854050de426" containerName="payload-processing" containerID="cri-o://0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80" gracePeriod=30
Jul 24 18:30:32.826701 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:32.826639    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"]
Jul 24 18:30:32.827805 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:32.827369    2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5" podUID="e6d97298-88bd-44f8-9713-7ebe61d53208" containerName="payload-pre-processing" containerID="cri-o://fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b" gracePeriod=30
Jul 24 18:30:33.113495 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.107801    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:30:33.118236 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.118208    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:30:33.219532 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.219435    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b90007-c72c-4883-a211-2854050de426-plugins-config-volume\") pod \"d0b90007-c72c-4883-a211-2854050de426\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") "
Jul 24 18:30:33.219532 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.219523    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np66p\" (UniqueName: \"kubernetes.io/projected/d0b90007-c72c-4883-a211-2854050de426-kube-api-access-np66p\") pod \"d0b90007-c72c-4883-a211-2854050de426\" (UID: \"d0b90007-c72c-4883-a211-2854050de426\") "
Jul 24 18:30:33.219788 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.219542    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d97298-88bd-44f8-9713-7ebe61d53208-plugins-config-volume\") pod \"e6d97298-88bd-44f8-9713-7ebe61d53208\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") "
Jul 24 18:30:33.219788 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.219566    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zv8p\" (UniqueName: \"kubernetes.io/projected/e6d97298-88bd-44f8-9713-7ebe61d53208-kube-api-access-4zv8p\") pod \"e6d97298-88bd-44f8-9713-7ebe61d53208\" (UID: \"e6d97298-88bd-44f8-9713-7ebe61d53208\") "
Jul 24 18:30:33.219940 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.219909    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0b90007-c72c-4883-a211-2854050de426-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0b90007-c72c-4883-a211-2854050de426" (UID: "d0b90007-c72c-4883-a211-2854050de426"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:30:33.220018 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.219951    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d97298-88bd-44f8-9713-7ebe61d53208-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6d97298-88bd-44f8-9713-7ebe61d53208" (UID: "e6d97298-88bd-44f8-9713-7ebe61d53208"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:30:33.221787 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.221758    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b90007-c72c-4883-a211-2854050de426-kube-api-access-np66p" (OuterVolumeSpecName: "kube-api-access-np66p") pod "d0b90007-c72c-4883-a211-2854050de426" (UID: "d0b90007-c72c-4883-a211-2854050de426"). InnerVolumeSpecName "kube-api-access-np66p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:30:33.221898 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.221798    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d97298-88bd-44f8-9713-7ebe61d53208-kube-api-access-4zv8p" (OuterVolumeSpecName: "kube-api-access-4zv8p") pod "e6d97298-88bd-44f8-9713-7ebe61d53208" (UID: "e6d97298-88bd-44f8-9713-7ebe61d53208"). InnerVolumeSpecName "kube-api-access-4zv8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 18:30:33.320635 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.320594    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np66p\" (UniqueName: \"kubernetes.io/projected/d0b90007-c72c-4883-a211-2854050de426-kube-api-access-np66p\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:33.320635 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.320628    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d97298-88bd-44f8-9713-7ebe61d53208-plugins-config-volume\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:33.320635 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.320637    2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zv8p\" (UniqueName: \"kubernetes.io/projected/e6d97298-88bd-44f8-9713-7ebe61d53208-kube-api-access-4zv8p\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:33.320635 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.320646    2575 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0b90007-c72c-4883-a211-2854050de426-plugins-config-volume\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:33.360041 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.359948    2575 generic.go:356] "Generic (PLEG): container finished" podID="d0b90007-c72c-4883-a211-2854050de426" containerID="0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80" exitCode=0
Jul 24 18:30:33.360041 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.360009    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"
Jul 24 18:30:33.360299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.360040    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9" event={"ID":"d0b90007-c72c-4883-a211-2854050de426","Type":"ContainerDied","Data":"0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80"}
Jul 24 18:30:33.360299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.360094    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9" event={"ID":"d0b90007-c72c-4883-a211-2854050de426","Type":"ContainerDied","Data":"b6a230fe9d4c4b53dce9987ca0cb6b144dc9bb0d8704a093dbb6587479fffafc"}
Jul 24 18:30:33.360299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.360123    2575 scope.go:117] "RemoveContainer" containerID="0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80"
Jul 24 18:30:33.361562 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.361537    2575 generic.go:356] "Generic (PLEG): container finished" podID="e6d97298-88bd-44f8-9713-7ebe61d53208" containerID="fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b" exitCode=0
Jul 24 18:30:33.361672 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.361575    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"
Jul 24 18:30:33.361672 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.361592    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5" event={"ID":"e6d97298-88bd-44f8-9713-7ebe61d53208","Type":"ContainerDied","Data":"fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b"}
Jul 24 18:30:33.361672 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.361632    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5" event={"ID":"e6d97298-88bd-44f8-9713-7ebe61d53208","Type":"ContainerDied","Data":"382cd084e5e93a140395a5ee5447b8b4b191fa68a7eefad42c44996957994c81"}
Jul 24 18:30:33.375281 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.375244    2575 scope.go:117] "RemoveContainer" containerID="0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80"
Jul 24 18:30:33.375717 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:30:33.375698    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80\": container with ID starting with 0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80 not found: ID does not exist" containerID="0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80"
Jul 24 18:30:33.375758 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.375729    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80"} err="failed to get container status \"0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80\": rpc error: code = NotFound desc = could not find container \"0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80\": container with ID starting with 0fa8a4721ed53d700ccb1a04c67224756c2686f19edb091edbd0929c3e335c80 not found: ID does not exist"
Jul 24 18:30:33.375758 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.375751    2575 scope.go:117] "RemoveContainer" containerID="fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b"
Jul 24 18:30:33.384659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.384634    2575 scope.go:117] "RemoveContainer" containerID="fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b"
Jul 24 18:30:33.385018 ip-10-0-140-161 kubenswrapper[2575]: E0724 18:30:33.384995    2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b\": container with ID starting with fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b not found: ID does not exist" containerID="fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b"
Jul 24 18:30:33.385090 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.385025    2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b"} err="failed to get container status \"fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b\": rpc error: code = NotFound desc = could not find container \"fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b\": container with ID starting with fae44322987d5307e0716c9af7cd1b3ea8095d9844eea21070691011d3e7c18b not found: ID does not exist"
Jul 24 18:30:33.387881 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.387853    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"]
Jul 24 18:30:33.390150 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.390128    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e18e67-58f69d47b-np2r5"]
Jul 24 18:30:33.401090 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.401057    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"]
Jul 24 18:30:33.405174 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:33.405144    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-e18e67-b66df5dd-7xcw9"]
Jul 24 18:30:34.137382 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:34.137348    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b90007-c72c-4883-a211-2854050de426" path="/var/lib/kubelet/pods/d0b90007-c72c-4883-a211-2854050de426/volumes"
Jul 24 18:30:34.137793 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:34.137713    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6d97298-88bd-44f8-9713-7ebe61d53208" path="/var/lib/kubelet/pods/e6d97298-88bd-44f8-9713-7ebe61d53208/volumes"
Jul 24 18:30:36.256995 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:36.256955    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:30:43.905034 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.904995    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"]
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905379    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905390    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905398    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6d97298-88bd-44f8-9713-7ebe61d53208" containerName="payload-pre-processing"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905404    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6d97298-88bd-44f8-9713-7ebe61d53208" containerName="payload-pre-processing"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905411    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905417    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905424    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905430    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905440    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905447    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905456    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aa567d8-e56a-493f-b028-cf3328b68452" containerName="main"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905461    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aa567d8-e56a-493f-b028-cf3328b68452" containerName="main"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905467    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee521783-2c52-482f-bc92-3fbbd9fe56d4" containerName="payload-pre-processing"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905472    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee521783-2c52-482f-bc92-3fbbd9fe56d4" containerName="payload-pre-processing"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905486    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b90007-c72c-4883-a211-2854050de426" containerName="payload-processing"
Jul 24 18:30:43.905503 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905491    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b90007-c72c-4883-a211-2854050de426" containerName="payload-processing"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905548    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b90007-c72c-4883-a211-2854050de426" containerName="payload-processing"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905556    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905561    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6d97298-88bd-44f8-9713-7ebe61d53208" containerName="payload-pre-processing"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905567    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905574    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="22fd428b-cfee-47fe-b72a-9038b3b17f3d" containerName="istio-proxy"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905580    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aa567d8-e56a-493f-b028-cf3328b68452" containerName="main"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905585    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb12923a-df24-43fb-81e1-28614f73ef25" containerName="istio-proxy"
Jul 24 18:30:43.905979 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.905589    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee521783-2c52-482f-bc92-3fbbd9fe56d4" containerName="payload-pre-processing"
Jul 24 18:30:43.910028 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.910010    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:43.914416 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.914396    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-v7cm8\""
Jul 24 18:30:43.914553 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.914397    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 18:30:43.922439 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:43.922402    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"]
Jul 24 18:30:44.005787 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:44.005749    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a29515f1-4006-4779-8037-313cd3336b21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd\" (UID: \"a29515f1-4006-4779-8037-313cd3336b21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:44.106485 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:44.106448    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a29515f1-4006-4779-8037-313cd3336b21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd\" (UID: \"a29515f1-4006-4779-8037-313cd3336b21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:44.107076 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:44.107056    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a29515f1-4006-4779-8037-313cd3336b21-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd\" (UID: \"a29515f1-4006-4779-8037-313cd3336b21\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:44.220603 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:44.220562    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:44.366653 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:44.366624    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"]
Jul 24 18:30:44.369011 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:30:44.368981    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29515f1_4006_4779_8037_313cd3336b21.slice/crio-65d70f46eb5435837252c4aed53a6b682c6368a1c223c0109a24076a7599fd28 WatchSource:0}: Error finding container 65d70f46eb5435837252c4aed53a6b682c6368a1c223c0109a24076a7599fd28: Status 404 returned error can't find the container with id 65d70f46eb5435837252c4aed53a6b682c6368a1c223c0109a24076a7599fd28
Jul 24 18:30:44.426735 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:44.426699    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd" event={"ID":"a29515f1-4006-4779-8037-313cd3336b21","Type":"ContainerStarted","Data":"65d70f46eb5435837252c4aed53a6b682c6368a1c223c0109a24076a7599fd28"}
Jul 24 18:30:47.445645 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:47.445602    2575 generic.go:356] "Generic (PLEG): container finished" podID="a29515f1-4006-4779-8037-313cd3336b21" containerID="2f96329c90bd7c28150907fc39820738cce43119490bd2fd2f25fd218e360d75" exitCode=0
Jul 24 18:30:47.446054 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:47.445676    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd" event={"ID":"a29515f1-4006-4779-8037-313cd3336b21","Type":"ContainerDied","Data":"2f96329c90bd7c28150907fc39820738cce43119490bd2fd2f25fd218e360d75"}
Jul 24 18:30:48.582659 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:48.582626    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:48.645699 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:48.645660    2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a29515f1-4006-4779-8037-313cd3336b21-maas-api-service-ca\") pod \"a29515f1-4006-4779-8037-313cd3336b21\" (UID: \"a29515f1-4006-4779-8037-313cd3336b21\") "
Jul 24 18:30:48.646057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:48.646033    2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a29515f1-4006-4779-8037-313cd3336b21-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a29515f1-4006-4779-8037-313cd3336b21" (UID: "a29515f1-4006-4779-8037-313cd3336b21"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 18:30:48.747299 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:48.747165    2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a29515f1-4006-4779-8037-313cd3336b21-maas-api-service-ca\") on node \"ip-10-0-140-161.ec2.internal\" DevicePath \"\""
Jul 24 18:30:49.456786 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:49.456746    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd" event={"ID":"a29515f1-4006-4779-8037-313cd3336b21","Type":"ContainerDied","Data":"65d70f46eb5435837252c4aed53a6b682c6368a1c223c0109a24076a7599fd28"}
Jul 24 18:30:49.456786 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:49.456779    2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"
Jul 24 18:30:49.457018 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:49.456808    2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65d70f46eb5435837252c4aed53a6b682c6368a1c223c0109a24076a7599fd28"
Jul 24 18:30:57.263401 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:57.263360    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:30:58.976780 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:58.976737    2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"]
Jul 24 18:30:58.980990 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:30:58.980954    2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-100cff-dfkzd"]
Jul 24 18:31:00.137560 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:00.137520    2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a29515f1-4006-4779-8037-313cd3336b21" path="/var/lib/kubelet/pods/a29515f1-4006-4779-8037-313cd3336b21/volumes"
Jul 24 18:31:19.860430 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:19.860388    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-wmlt9"]
Jul 24 18:31:35.052997 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:35.052956    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b794f588d-rcgmt_468882ed-2cc3-458a-8fde-48d9fb4e8c38/copy-manifests/0.log"
Jul 24 18:31:35.060475 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:35.060449    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b794f588d-rcgmt_468882ed-2cc3-458a-8fde-48d9fb4e8c38/manager/0.log"
Jul 24 18:31:37.533898 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:37.533849    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nql2s_25037198-0876-44b9-ac61-d04943afaf77/manager/0.log"
Jul 24 18:31:37.657800 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:37.657768    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x426q_167b8179-0965-444d-8055-f9cbdea5aa6d/kuadrant-console-plugin/0.log"
Jul 24 18:31:38.013804 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:38.013775    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wmlt9_f952578c-31b7-4e67-bb27-1d402585b995/limitador/0.log"
Jul 24 18:31:38.128059 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:38.128023    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k6kq5_40d7aa2b-b0e0-4d2d-a30d-745d2abc849d/manager/0.log"
Jul 24 18:31:38.844294 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:38.844240    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546fb58d4c-7fdxd_92b081b3-0862-4445-8f79-9adadfd063c6/kube-auth-proxy/0.log"
Jul 24 18:31:39.347185 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:39.347138    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657bdf55cc-7jqkw_fba53f5c-2b5b-4b91-abed-426505542cc9/router/0.log"
Jul 24 18:31:40.259644 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:40.259615    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-6t72m_e89eed6d-01c6-40f5-9f30-b746c1e8b19f/main/0.log"
Jul 24 18:31:42.331121 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:42.331093    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:31:42.340622 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:42.340597    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:31:45.601206 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.601166    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwr8f/must-gather-9dhjh"]
Jul 24 18:31:45.601661 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.601574    2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29515f1-4006-4779-8037-313cd3336b21" containerName="revoke-keys"
Jul 24 18:31:45.601661 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.601592    2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29515f1-4006-4779-8037-313cd3336b21" containerName="revoke-keys"
Jul 24 18:31:45.601734 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.601670    2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a29515f1-4006-4779-8037-313cd3336b21" containerName="revoke-keys"
Jul 24 18:31:45.605336 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.605311    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.607435 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.607410    2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dwr8f\"/\"default-dockercfg-85vn6\""
Jul 24 18:31:45.608057 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.608036    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwr8f\"/\"kube-root-ca.crt\""
Jul 24 18:31:45.608181 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.608098    2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dwr8f\"/\"openshift-service-ca.crt\""
Jul 24 18:31:45.615090 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.615060    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwr8f/must-gather-9dhjh"]
Jul 24 18:31:45.686330 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.686259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz6r7\" (UniqueName: \"kubernetes.io/projected/74035ae1-cb0f-4ec6-9038-87c7573770a8-kube-api-access-gz6r7\") pod \"must-gather-9dhjh\" (UID: \"74035ae1-cb0f-4ec6-9038-87c7573770a8\") " pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.686330 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.686334    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74035ae1-cb0f-4ec6-9038-87c7573770a8-must-gather-output\") pod \"must-gather-9dhjh\" (UID: \"74035ae1-cb0f-4ec6-9038-87c7573770a8\") " pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.787496 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.787455    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz6r7\" (UniqueName: \"kubernetes.io/projected/74035ae1-cb0f-4ec6-9038-87c7573770a8-kube-api-access-gz6r7\") pod \"must-gather-9dhjh\" (UID: \"74035ae1-cb0f-4ec6-9038-87c7573770a8\") " pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.787701 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.787502    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74035ae1-cb0f-4ec6-9038-87c7573770a8-must-gather-output\") pod \"must-gather-9dhjh\" (UID: \"74035ae1-cb0f-4ec6-9038-87c7573770a8\") " pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.787947 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.787927    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74035ae1-cb0f-4ec6-9038-87c7573770a8-must-gather-output\") pod \"must-gather-9dhjh\" (UID: \"74035ae1-cb0f-4ec6-9038-87c7573770a8\") " pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.796740 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.796704    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz6r7\" (UniqueName: \"kubernetes.io/projected/74035ae1-cb0f-4ec6-9038-87c7573770a8-kube-api-access-gz6r7\") pod \"must-gather-9dhjh\" (UID: \"74035ae1-cb0f-4ec6-9038-87c7573770a8\") " pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:45.915645 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:45.915544    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwr8f/must-gather-9dhjh"
Jul 24 18:31:46.059890 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:46.059859    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwr8f/must-gather-9dhjh"]
Jul 24 18:31:46.062631 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:31:46.062597    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74035ae1_cb0f_4ec6_9038_87c7573770a8.slice/crio-7e9cfad14d1979a438a435eb23f44c6991e240b50829788743be45a9a5b49af6 WatchSource:0}: Error finding container 7e9cfad14d1979a438a435eb23f44c6991e240b50829788743be45a9a5b49af6: Status 404 returned error can't find the container with id 7e9cfad14d1979a438a435eb23f44c6991e240b50829788743be45a9a5b49af6
Jul 24 18:31:46.064517 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:46.064497    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 18:31:46.744904 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:46.744866    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwr8f/must-gather-9dhjh" event={"ID":"74035ae1-cb0f-4ec6-9038-87c7573770a8","Type":"ContainerStarted","Data":"7e9cfad14d1979a438a435eb23f44c6991e240b50829788743be45a9a5b49af6"}
Jul 24 18:31:47.758227 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:47.757538    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwr8f/must-gather-9dhjh" event={"ID":"74035ae1-cb0f-4ec6-9038-87c7573770a8","Type":"ContainerStarted","Data":"c88328429aad3a128ed19e13a06061ca36967ec22175b2402075228793d599ef"}
Jul 24 18:31:47.758227 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:47.757599    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwr8f/must-gather-9dhjh" event={"ID":"74035ae1-cb0f-4ec6-9038-87c7573770a8","Type":"ContainerStarted","Data":"2d586086f451e61d489ae14213940e7fc4159c81802388062281e4f67873d41f"}
Jul 24 18:31:47.778052 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:47.777962    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwr8f/must-gather-9dhjh" podStartSLOduration=2.051752493 podStartE2EDuration="2.777937769s" totalImagesPullingTime="726.185276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 18:31:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:31:47.774539626 +0000 UTC m=+3306.178480877" watchObservedRunningTime="2026-07-24 18:31:47.777937769 +0000 UTC m=+3306.181879000"
Jul 24 18:31:48.511424 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:48.511390    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-57pv5_76828591-503d-4d58-87cb-a570db5fc73f/global-pull-secret-syncer/0.log"
Jul 24 18:31:48.759727 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:48.759692    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ksf6l_fde4c405-d38a-4578-ba07-de3466e88cb0/konnectivity-agent/0.log"
Jul 24 18:31:48.811926 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:48.811841    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-161.ec2.internal_42c822a09c30d0b2448c94638a327500/haproxy/0.log"
Jul 24 18:31:54.858231 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:54.858187    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-nql2s_25037198-0876-44b9-ac61-d04943afaf77/manager/0.log"
Jul 24 18:31:54.888516 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:54.888476    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x426q_167b8179-0965-444d-8055-f9cbdea5aa6d/kuadrant-console-plugin/0.log"
Jul 24 18:31:55.143297 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:55.143190    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-wmlt9_f952578c-31b7-4e67-bb27-1d402585b995/limitador/0.log"
Jul 24 18:31:55.247815 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:55.247784    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-k6kq5_40d7aa2b-b0e0-4d2d-a30d-745d2abc849d/manager/0.log"
Jul 24 18:31:57.211607 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.211534    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-jl7tq_6e1d3688-4227-4f08-90ab-642c12dd03fd/kube-state-metrics/0.log"
Jul 24 18:31:57.237058 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.237019    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-jl7tq_6e1d3688-4227-4f08-90ab-642c12dd03fd/kube-rbac-proxy-main/0.log"
Jul 24 18:31:57.261213 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.261183    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-jl7tq_6e1d3688-4227-4f08-90ab-642c12dd03fd/kube-rbac-proxy-self/0.log"
Jul 24 18:31:57.291750 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.291719    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6b47d57b47-hv7vd_7d521f2a-e3f8-48a6-95f2-77015e928da1/metrics-server/0.log"
Jul 24 18:31:57.318238 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.318202    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-2sqqw_ac4e5bd8-4f8c-40b6-8884-82d0d0b6eb1d/monitoring-plugin/0.log"
Jul 24 18:31:57.504336 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.504200    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v6jgr_71cbfa0f-d3dc-43c9-9488-8a3953536703/node-exporter/0.log"
Jul 24 18:31:57.527379 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.527352    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v6jgr_71cbfa0f-d3dc-43c9-9488-8a3953536703/kube-rbac-proxy/0.log"
Jul 24 18:31:57.563030 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.563001    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v6jgr_71cbfa0f-d3dc-43c9-9488-8a3953536703/init-textfile/0.log"
Jul 24 18:31:57.676245 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.676207    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/prometheus/0.log"
Jul 24 18:31:57.697867 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.697836    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/config-reloader/0.log"
Jul 24 18:31:57.723192 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.723162    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/thanos-sidecar/0.log"
Jul 24 18:31:57.748571 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.748544    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/kube-rbac-proxy-web/0.log"
Jul 24 18:31:57.771830 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.771731    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/kube-rbac-proxy/0.log"
Jul 24 18:31:57.795293 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.795248    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/kube-rbac-proxy-thanos/0.log"
Jul 24 18:31:57.817287 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.817239    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9e6ae60c-a0da-4fce-9ece-94209736f20d/init-config-reloader/0.log"
Jul 24 18:31:57.851056 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.851021    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-9n9kc_a46d6ad4-81ac-4892-8211-ca55bc65d596/prometheus-operator/0.log"
Jul 24 18:31:57.871079 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:57.871045    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-9n9kc_a46d6ad4-81ac-4892-8211-ca55bc65d596/kube-rbac-proxy/0.log"
Jul 24 18:31:58.001712 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.001678    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c94485f49-5xcwq_ca9318da-5d7b-4901-83a2-d5b53607450f/thanos-query/0.log"
Jul 24 18:31:58.025390 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.025304    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c94485f49-5xcwq_ca9318da-5d7b-4901-83a2-d5b53607450f/kube-rbac-proxy-web/0.log"
Jul 24 18:31:58.049726 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.049695    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c94485f49-5xcwq_ca9318da-5d7b-4901-83a2-d5b53607450f/kube-rbac-proxy/0.log"
Jul 24 18:31:58.076861 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.076833    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c94485f49-5xcwq_ca9318da-5d7b-4901-83a2-d5b53607450f/prom-label-proxy/0.log"
Jul 24 18:31:58.099483 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.099455    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c94485f49-5xcwq_ca9318da-5d7b-4901-83a2-d5b53607450f/kube-rbac-proxy-rules/0.log"
Jul 24 18:31:58.122580 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.122551    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c94485f49-5xcwq_ca9318da-5d7b-4901-83a2-d5b53607450f/kube-rbac-proxy-metrics/0.log"
Jul 24 18:31:58.795256 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.795215    2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"]
Jul 24 18:31:58.800239 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.800204    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:58.809923 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.809891    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"]
Jul 24 18:31:58.951591 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.951553    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-podres\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:58.951791 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.951640    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-proc\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:58.951791 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.951689    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-sys\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:58.951791 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.951746    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-lib-modules\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:58.951947 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:58.951811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf46p\" (UniqueName: \"kubernetes.io/projected/4f088a14-d004-436b-a347-3e80ebe62937-kube-api-access-lf46p\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.052835 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.052726    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-proc\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.052835 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.052811    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-sys\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053101 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.052851    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-lib-modules\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053101 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.052888    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf46p\" (UniqueName: \"kubernetes.io/projected/4f088a14-d004-436b-a347-3e80ebe62937-kube-api-access-lf46p\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053101 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.052961    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-podres\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053286 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.053140    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-podres\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053286 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.053206    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-proc\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053286 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.053251    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-sys\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.053461 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.053367    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4f088a14-d004-436b-a347-3e80ebe62937-lib-modules\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.063627 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.063582    2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf46p\" (UniqueName: \"kubernetes.io/projected/4f088a14-d004-436b-a347-3e80ebe62937-kube-api-access-lf46p\") pod \"perf-node-gather-daemonset-d7bdg\" (UID: \"4f088a14-d004-436b-a347-3e80ebe62937\") " pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.117212 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.117163    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.316335 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.316214    2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"]
Jul 24 18:31:59.321662 ip-10-0-140-161 kubenswrapper[2575]: W0724 18:31:59.321158    2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4f088a14_d004_436b_a347_3e80ebe62937.slice/crio-578d02716b5a45c651f4379860c07f7b8efbac892166866fe3705348f52ad48f WatchSource:0}: Error finding container 578d02716b5a45c651f4379860c07f7b8efbac892166866fe3705348f52ad48f: Status 404 returned error can't find the container with id 578d02716b5a45c651f4379860c07f7b8efbac892166866fe3705348f52ad48f
Jul 24 18:31:59.846990 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.846946    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg" event={"ID":"4f088a14-d004-436b-a347-3e80ebe62937","Type":"ContainerStarted","Data":"650a7dacf67ad4f3aa22b9c9713276f9b2cc08945533186d82c58e1411f08254"}
Jul 24 18:31:59.847498 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.847015    2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg" event={"ID":"4f088a14-d004-436b-a347-3e80ebe62937","Type":"ContainerStarted","Data":"578d02716b5a45c651f4379860c07f7b8efbac892166866fe3705348f52ad48f"}
Jul 24 18:31:59.847498 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.847132    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:31:59.863916 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.863856    2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg" podStartSLOduration=1.863840376 podStartE2EDuration="1.863840376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 18:31:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 18:31:59.863181958 +0000 UTC m=+3318.267123188" watchObservedRunningTime="2026-07-24 18:31:59.863840376 +0000 UTC m=+3318.267781605"
Jul 24 18:31:59.971454 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.971413    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/1.log"
Jul 24 18:31:59.977570 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:31:59.977514    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-2lg9d_f5de17fc-1e9b-44c7-9096-8b419c33c108/console-operator/2.log"
Jul 24 18:32:02.475662 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:02.475629    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sfqpz_f3042e4e-4106-4f3c-be76-9677db69bbf2/dns/0.log"
Jul 24 18:32:02.499754 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:02.499724    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-sfqpz_f3042e4e-4106-4f3c-be76-9677db69bbf2/kube-rbac-proxy/0.log"
Jul 24 18:32:02.526277 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:02.526224    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-59zmp_3855ca0f-6f2c-4fbc-b018-b5b0033ae11d/dns-node-resolver/0.log"
Jul 24 18:32:03.239469 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:03.239338    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gtbnj_fe83c62b-750b-4ae3-a4ae-5ca6a056c125/node-ca/0.log"
Jul 24 18:32:04.400546 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:04.400508    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546fb58d4c-7fdxd_92b081b3-0862-4445-8f79-9adadfd063c6/kube-auth-proxy/0.log"
Jul 24 18:32:04.584284 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:04.584234    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657bdf55cc-7jqkw_fba53f5c-2b5b-4b91-abed-426505542cc9/router/0.log"
Jul 24 18:32:05.459703 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:05.459672    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tzltv_07bacd15-46fd-4cdf-b7a1-02ae5ff7086c/serve-healthcheck-canary/0.log"
Jul 24 18:32:05.871652 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:05.871559    2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dwr8f/perf-node-gather-daemonset-d7bdg"
Jul 24 18:32:05.979237 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:05.979205    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-rcfqt_970c6912-05fb-4fd7-a428-407244b542fc/insights-operator/0.log"
Jul 24 18:32:05.979722 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:05.979704    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-rcfqt_970c6912-05fb-4fd7-a428-407244b542fc/insights-operator/1.log"
Jul 24 18:32:06.076795 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:06.076760    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4n5p_7e81d826-1445-4170-b886-0f36ac0b5d00/kube-rbac-proxy/0.log"
Jul 24 18:32:06.099706 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:06.099671    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4n5p_7e81d826-1445-4170-b886-0f36ac0b5d00/exporter/0.log"
Jul 24 18:32:06.121439 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:06.121395    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4n5p_7e81d826-1445-4170-b886-0f36ac0b5d00/extractor/0.log"
Jul 24 18:32:09.450746 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:09.450712    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b794f588d-rcgmt_468882ed-2cc3-458a-8fde-48d9fb4e8c38/manager/0.log"
Jul 24 18:32:09.476722 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:09.476630    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b794f588d-rcgmt_468882ed-2cc3-458a-8fde-48d9fb4e8c38/copy-manifests/0.log"
Jul 24 18:32:10.433052 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:10.433025    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546fb58d4c-7fdxd_92b081b3-0862-4445-8f79-9adadfd063c6/kube-auth-proxy/0.log"
Jul 24 18:32:10.630905 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:10.630868    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657bdf55cc-7jqkw_fba53f5c-2b5b-4b91-abed-426505542cc9/router/0.log"
Jul 24 18:32:11.013847 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:11.013771    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b4bb5cddb-bcptj_77ebb0f4-a143-49b1-8a5f-3ff538c5a6e7/manager/0.log"
Jul 24 18:32:12.365021 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:12.364991    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546fb58d4c-7fdxd_92b081b3-0862-4445-8f79-9adadfd063c6/kube-auth-proxy/0.log"
Jul 24 18:32:12.564022 ip-10-0-140-161 kubenswrapper[2575]: I0724 18:32:12.563985    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-657bdf55cc-7jqkw_fba53f5c-2b5b-4b91-abed-426505542cc9/router/0.log"