Jul 14 22:07:13.097908 ip-10-0-143-117 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 14 22:07:13.097922 ip-10-0-143-117 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 14 22:07:13.097932 ip-10-0-143-117 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 14 22:07:13.098415 ip-10-0-143-117 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 14 22:07:23.171151 ip-10-0-143-117 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 14 22:07:23.171171 ip-10-0-143-117 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8e04847428644065aae4546a5c83c060 --
Jul 14 22:09:19.005400 ip-10-0-143-117 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 22:09:19.372840 ip-10-0-143-117 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 22:09:19.372840 ip-10-0-143-117 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 22:09:19.372840 ip-10-0-143-117 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 22:09:19.372840 ip-10-0-143-117 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 22:09:19.372840 ip-10-0-143-117 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 22:09:19.374024 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.373877    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 22:09:19.378851 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378833    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 22:09:19.378851 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378851    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378855    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378858    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378862    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378865    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378868    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378871    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378874    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378878    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378881    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378883    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378886    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378889    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378892    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378894    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378897    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378900    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378903    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378906    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378908    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 22:09:19.378929 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378913    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378915    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378918    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378921    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378923    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378926    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378929    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378931    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378934    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378939    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378944    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378949    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378952    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378955    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378958    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378961    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378964    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378967    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378970    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 22:09:19.379414 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378972    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378975    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378978    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378981    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378984    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378986    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378989    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378992    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378996    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.378998    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379001    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379004    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379007    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379009    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379017    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379021    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379023    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379026    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379029    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 22:09:19.379871 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379033    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379037    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379040    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379043    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379046    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379049    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379052    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379055    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379059    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379062    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379065    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379067    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379070    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379072    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379075    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379077    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379080    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379119    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379124    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379127    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 22:09:19.380333 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379129    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379132    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379135    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379138    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379141    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379144    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379146    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379148    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379153    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379157    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379159    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379162    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379165    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379167    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379170    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379173    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379175    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379178    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379180    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379184    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 22:09:19.380792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379914    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379922    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379925    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379928    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379931    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379933    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379936    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379939    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379941    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379944    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379947    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379950    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379953    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379956    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379958    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379961    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379964    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379967    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379970    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 22:09:19.381318 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379973    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379977    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379980    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379983    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379985    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379988    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379991    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379993    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379995    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.379998    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380000    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380003    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380005    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380008    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380011    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380014    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380016    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380019    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380021    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380024    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 22:09:19.381792 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380026    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380029    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380031    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380034    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380037    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380039    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380042    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380045    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380047    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380050    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380052    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380055    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380057    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380061    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380064    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380067    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380070    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380073    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380076    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380078    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 22:09:19.382298 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380081    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380084    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380086    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380089    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380091    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380113    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380117    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380121    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380125    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380129    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380132    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380135    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380138    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380142    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380144    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380148    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380152    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380156    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 22:09:19.382781 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380160    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380162    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380165    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380168    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380170    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380173    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380176    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380179    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380182    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380185    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380190    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380193    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380195    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380198    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380201    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380203    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380206    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380208    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380211    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380214    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 22:09:19.383239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380217    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380220    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380681    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380689    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380693    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380696    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380699    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380702    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380705    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380708    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380711    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380715    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380717    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380720    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380723    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380725    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380728    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380731    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380733    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380737    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 22:09:19.383738 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380740    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380743    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380749    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380752    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380755    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380757    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380760    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380763    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380766    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380768    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380770    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380773    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380776    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380778    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380782    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380784    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380787    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380790    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380792    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380795    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 22:09:19.384223 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380799    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380802    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380804    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380807    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380810    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380813    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380815    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380818    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380821    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380823    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380827    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380830    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380833    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380836    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380838    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380841    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380844    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380846    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380849    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380852    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 22:09:19.384686 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380854    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380857    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380859    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380862    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380864    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380867    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380869    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380872    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380875    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380877    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380882    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380885    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380888    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380891    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380893    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380897    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380899    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380901    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380904    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 22:09:19.385200 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380906    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380910    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380913    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380916    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380919    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380922    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380925    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380927    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380931    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380934    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380937    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380939    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380942    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380944    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380947    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380950    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380952    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380955    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 22:09:19.385666 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380957    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380961    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380964    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.380967    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381039    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381046    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381053    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381057    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381062    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381065    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381070    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381074    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381077    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381080    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381084    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381087    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381090    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381093    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381110    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381113    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381117    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381120    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381124    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 14 22:09:19.386129 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381127    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381131    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381134    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381138    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381142    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381145    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381148    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381151    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381155    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381158    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381161    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381164    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381167    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381171    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381175    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381178    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381181    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381184    2580 flags.go:64] FLAG: --enable-server="true"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381187    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381193    2580 flags.go:64] FLAG: --event-burst="100"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381197    2580 flags.go:64] FLAG: --event-qps="50"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381200    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381204    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381207    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 14 22:09:19.386713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381210    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381213    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381216    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381219    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381223    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381226    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381229    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381231    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381234    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381238    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381242    2580 flags.go:64] FLAG: --feature-gates=""
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381246    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381249    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381252    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381255    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381259    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381262    2580 flags.go:64] FLAG: --help="false"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381265    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381268    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381272    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381275    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381279    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381282    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381285    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 22:09:19.387284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381288    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381291    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381294    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381298    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381301    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381304    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381307    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381309    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381312    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381315    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381318    2580 flags.go:64] FLAG: --lock-file=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381321    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381325    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381328    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381333    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381336    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381339    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381343    2580 flags.go:64] FLAG: --logging-format="text"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381346    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381349    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381352    2580 flags.go:64] FLAG: --manifest-url=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381355    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381360    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381363    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381368    2580 flags.go:64] FLAG: --max-pods="110"
Jul 14 22:09:19.387904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381371    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381374    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381379    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381383    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381386    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381389    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381392    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381400    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381403    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381407    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381413    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381416    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381423    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381426    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381429    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381432    2580 flags.go:64] FLAG: --port="10250"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381435    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381438    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b21c4ad2a817dd3c"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381442    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381445    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381448    2580 flags.go:64] FLAG: --register-node="true"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381451    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381455    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381458    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 14 22:09:19.388600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381461    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381464    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381468    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381472    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381475    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381478    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381481    2580 flags.go:64] FLAG: --runonce="false"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381484    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381487    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381495    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381502    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381505    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381508    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381512    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381515    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381518    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381521    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381524    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381527    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381540    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381544    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381547    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381552    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381555    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381558    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 22:09:19.389556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381563    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381565    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381568    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381571    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381574    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381577    2580 flags.go:64] FLAG: --v="2"
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381582    2580 flags.go:64] FLAG: --version="false"
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381586    2580 flags.go:64] FLAG: --vmodule=""
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381589    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.381593    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381729    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381735    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381738    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381741    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381744    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381747    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381750    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381754    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381757    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381761    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381764    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381767    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 22:09:19.390269 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381770    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381773    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381776    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381779    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381783    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381786    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381788    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381792    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381797    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381800    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381803    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381807    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381810    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381812    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381815    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381817    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381820    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381823    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381825    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 22:09:19.390828 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381828    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381831    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381833    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381837    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381839    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381842    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381845    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381848    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381851    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381854    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381857    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381859    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381862    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381864    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381867    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381870    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381872    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381876    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381879    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381881    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 22:09:19.391309 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381884    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381887    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381889    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381893    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381897    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381900    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381903    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381905    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381908    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381910    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381913    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381916    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381919    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381921    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381924    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381927    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381930    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381932    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381935    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 22:09:19.391802 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381938    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381942    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381944    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381947    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381949    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381952    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381955    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381957    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381960    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381962    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381966    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381969    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381971    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381974    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381976    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381979    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381981    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381984    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381986    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381989    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 22:09:19.392282 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381992    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381995    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.381997    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.382000    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.382003    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.382005    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.382008    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.382010    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.382013    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.382586    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.391059    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.391078    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.391111    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.391120    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391181    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 22:09:19.392769 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391186    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391190    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391193    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391196    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391199    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391203    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391205    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391208    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391211    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391213    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391216    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391219    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391222    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391224    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391227    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391229    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391234    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391238    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391242    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 22:09:19.393154 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391245    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391248    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391251    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391253    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391256    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391259    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391262    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391265    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391267    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391270    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391274    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391277    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391279    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391282    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391285    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391288    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391290    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391293    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391296    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391298    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 22:09:19.393616 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391301    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391304    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391306    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391309    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391312    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391315    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391319    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391321    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391325    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391329    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391332    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391334    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391337    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391339    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391342    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391345    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391348    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391350    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391353    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391355    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 22:09:19.394093 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391358    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391362    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391364    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391367    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391370    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391373    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391376    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391378    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391381    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391384    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391386    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391389    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391392    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391394    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391397    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391399    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391402    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391405    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391407    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 22:09:19.394590 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391412    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391416    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391419    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391422    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391425    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391428    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391431    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391434    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391437    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391439    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391442    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391445    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391447    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391450    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391453    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391456    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391458    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391461    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391463    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 22:09:19.395043 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391466    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.391471    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391595    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391601    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391604    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391607    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391610    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391613    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391616    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391619    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391622    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391624    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391627    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391631    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391635    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 22:09:19.395598 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391638    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391641    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391644    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391647    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391651    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391654    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391656    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391659    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391662    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391665    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391668    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391670    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391673    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391676    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391679    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391682    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391684    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391687    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391690    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391693    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 22:09:19.395960 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391695    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391697    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391700    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391702    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391705    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391708    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391711    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391713    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391716    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391719    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391721    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391724    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391727    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391730    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391732    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391735    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391738    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391740    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391743    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 22:09:19.396449 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391745    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391749    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391753    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391755    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391758    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391761    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391764    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391767    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391770    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391773    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391776    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391779    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391782    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391785    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391787    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391790    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391793    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391796    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391798    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 22:09:19.397018 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391801    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391804    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391808    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391811    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391814    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391817    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391820    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391824    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391826    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391829    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391832    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391834    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391837    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391839    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391842    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391844    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391847    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391849    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391852    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391855    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 22:09:19.397483 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391858    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391861    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391863    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391866    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391869    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391871    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391874    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:19.391877    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.391882    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.392016    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.394174    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.395400    2580 server.go:1019] "Starting client certificate rotation"
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.395515    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.395589    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 22:09:19.397975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.397820    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 22:09:19.399840 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.399826    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 22:09:19.410735 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.410716    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 14 22:09:19.413142 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.413128    2580 log.go:25] "Validated CRI v1 image API"
Jul 14 22:09:19.414569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.414555    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 22:09:19.420155 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.420137    2580 fs.go:135] Filesystem UUIDs: map[5797da73-a0c2-43dd-b1c9-0321a3a948d9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e347b21b-1d1e-4307-b76d-7d498a576e24:/dev/nvme0n1p4]
Jul 14 22:09:19.420208 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.420155    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 22:09:19.425466 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.425443    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 22:09:19.426183 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.425862    2580 manager.go:216] Machine: {Timestamp:2026-07-14 22:09:19.424318499 +0000 UTC m=+0.310164070 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094697 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec280620fcb7c2a30b1c0ac72ef4da42 SystemUUID:ec280620-fcb7-c2a3-0b1c-0ac72ef4da42 BootID:8e048474-2864-4065-aae4-546a5c83c060 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3b:44:60:44:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3b:44:60:44:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:56:aa:e4:69:b2:d0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 22:09:19.426183 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.426182    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 22:09:19.426289 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.426276    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 22:09:19.429238 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.429211    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 22:09:19.429368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.429242    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-117.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 22:09:19.429882 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.429872    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 22:09:19.429916 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.429884    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 22:09:19.429916 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.429892    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 22:09:19.430628 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.430617    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 22:09:19.431375 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.431365    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 22:09:19.433989 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.433978    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 22:09:19.434131 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434121    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 22:09:19.434173 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434153    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 22:09:19.434173 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434163    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 22:09:19.434173 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434174    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 22:09:19.434256 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434183    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 14 22:09:19.434256 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434192    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 22:09:19.434633 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434614    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 22:09:19.434674 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.434638    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 22:09:19.437358 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.437338    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 22:09:19.438682 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.438668    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 22:09:19.438730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.438698    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 22:09:19.439299 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439288    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 22:09:19.439357 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439304    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 22:09:19.439357 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439310    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 22:09:19.439357 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439316    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 22:09:19.439357 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439325    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 22:09:19.439357 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439330    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 22:09:19.439564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439463    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 22:09:19.439564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439469    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 22:09:19.439564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439477    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 22:09:19.439564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439483    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 22:09:19.439564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439492    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 22:09:19.439564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.439500    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 22:09:19.440516 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.440502    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 22:09:19.440558 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.440519    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 22:09:19.444140 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.444126    2580 server.go:1268] "Started kubelet"
Jul 14 22:09:19.444388 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.444321    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 22:09:19.444388 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.444281    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 22:09:19.444541 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.444411    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 22:09:19.445210 ip-10-0-143-117 systemd[1]: Started Kubernetes Kubelet.
Jul 14 22:09:19.445829 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.445663    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 22:09:19.446668 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.446640    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 22:09:19.446788 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.446764    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 22:09:19.446848 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.446778    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 22:09:19.446848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.446762    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 22:09:19.450921 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.450900    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 22:09:19.451761 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.451744    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 22:09:19.451841 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.451757    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 22:09:19.451967 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.451953    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:19.452478 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.451374    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c247727d2d3f05  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-14 22:09:19.444098821 +0000 UTC m=+0.329944390,LastTimestamp:2026-07-14 22:09:19.444098821 +0000 UTC m=+0.329944390,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 14 22:09:19.452478 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452343    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n6gvs"
Jul 14 22:09:19.452652 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452540    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 22:09:19.452652 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452557    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 22:09:19.452742 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452689    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 22:09:19.452806 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452784    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 22:09:19.452806 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452801    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 22:09:19.452935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452851    2580 factory.go:55] Registering systemd factory
Jul 14 22:09:19.452935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.452862    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 14 22:09:19.453086 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.453069    2580 factory.go:153] Registering CRI-O factory
Jul 14 22:09:19.453160 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.453090    2580 factory.go:223] Registration of the crio container factory successfully
Jul 14 22:09:19.453194 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.453168    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 22:09:19.453242 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.453198    2580 factory.go:103] Registering Raw factory
Jul 14 22:09:19.453242 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.453214    2580 manager.go:1196] Started watching for new ooms in manager
Jul 14 22:09:19.453955 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.453933    2580 manager.go:349] Starting recovery of all containers
Jul 14 22:09:19.460008 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.459956    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 22:09:19.460164 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.460114    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 22:09:19.462372 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.462294    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 22:09:19.462542 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.462388    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n6gvs"
Jul 14 22:09:19.467055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.466849    2580 manager.go:354] Recovery completed
Jul 14 22:09:19.472136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.472122    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.474842 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.474351    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.474842 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.474387    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.474842 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.474404    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.475564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.475541    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 22:09:19.475564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.475563    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 22:09:19.475711 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.475600    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 22:09:19.475803 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.475783    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.476704 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.476677    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.476791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.476717    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.476791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.476732    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.476791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.476739    2580 policy_none.go:49] "None policy: Start"
Jul 14 22:09:19.476791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.476756    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 22:09:19.476791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.476770    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 22:09:19.476939 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.476749    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-117.ec2.internal.18c247727efb2bd6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-117.ec2.internal,UID:ip-10-0-143-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-117.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-117.ec2.internal,},FirstTimestamp:2026-07-14 22:09:19.474371542 +0000 UTC m=+0.360217117,LastTimestamp:2026-07-14 22:09:19.474371542 +0000 UTC m=+0.360217117,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-117.ec2.internal,}"
Jul 14 22:09:19.477820 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.477808    2580 policy_none.go:47] "Start"
Jul 14 22:09:19.511203 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511169    2580 manager.go:337] "Starting Device Plugin manager"
Jul 14 22:09:19.511310 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.511243    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 22:09:19.511310 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511258    2580 server.go:87] "Starting device plugin registration server"
Jul 14 22:09:19.511571 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511555    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 22:09:19.511633 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511577    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 22:09:19.511796 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511770    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 22:09:19.511897 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511859    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 22:09:19.511897 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.511869    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 22:09:19.512465 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.512433    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 22:09:19.512570 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.512487    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:19.594476 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.594447    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 22:09:19.595665 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.594491    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 22:09:19.595665 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.594517    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 22:09:19.595665 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.594561    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 22:09:19.596909 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.596892    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 22:09:19.612438 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.612410    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.613362 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.613345    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.613432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.613382    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.613432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.613396    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.613432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.613425    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.625789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.625733    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.625789 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.625761    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-117.ec2.internal\": node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:19.674546 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.674512    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:19.695301 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.695262    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"]
Jul 14 22:09:19.695385 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.695346    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.696965 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.696937    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.697092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.696977    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.697092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.696992    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.698564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.698549    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.698696 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.698681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.698735 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.698713    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.699329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.699308    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.699439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.699339    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.699439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.699353    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.699439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.699311    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.699439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.699382    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.699439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.699398    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.700687 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.700670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.700738 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.700700    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 22:09:19.701454 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.701438    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 22:09:19.701553 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.701473    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 22:09:19.701553 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.701489    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeHasSufficientPID"
Jul 14 22:09:19.715790 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.715769    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-117.ec2.internal\" not found" node="ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.721014 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.720997    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-117.ec2.internal\" not found" node="ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.755155 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.755120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a03d3c4180aa732023cfc58b2079a1b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"a03d3c4180aa732023cfc58b2079a1b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.755266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.755162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a03d3c4180aa732023cfc58b2079a1b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"a03d3c4180aa732023cfc58b2079a1b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.755266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.755182    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55527337be11eb05926e7cca822eb38f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-117.ec2.internal\" (UID: \"55527337be11eb05926e7cca822eb38f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.774674 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.774645    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:19.856067 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.856028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a03d3c4180aa732023cfc58b2079a1b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"a03d3c4180aa732023cfc58b2079a1b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.856067 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.856071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a03d3c4180aa732023cfc58b2079a1b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"a03d3c4180aa732023cfc58b2079a1b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.856316 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.856092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55527337be11eb05926e7cca822eb38f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-117.ec2.internal\" (UID: \"55527337be11eb05926e7cca822eb38f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.856316 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.856164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/55527337be11eb05926e7cca822eb38f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-117.ec2.internal\" (UID: \"55527337be11eb05926e7cca822eb38f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.856316 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.856165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a03d3c4180aa732023cfc58b2079a1b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"a03d3c4180aa732023cfc58b2079a1b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.856316 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:19.856180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a03d3c4180aa732023cfc58b2079a1b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal\" (UID: \"a03d3c4180aa732023cfc58b2079a1b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:19.875724 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.875693    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:19.976010 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:19.975938    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.019282 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.019250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:20.024164 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.024141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:20.076208 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.076160    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.177278 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.177244    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.278213 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.278123    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.287975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.287944    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 22:09:20.379022 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.378977    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.395365 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.395329    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 22:09:20.395545 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.395520    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 22:09:20.395608 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.395550    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 22:09:20.467789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.467744    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 22:04:19 +0000 UTC" deadline="2028-02-23 08:22:21.944324663 +0000 UTC"
Jul 14 22:09:20.467789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.467788    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14122h13m1.476542476s"
Jul 14 22:09:20.479755 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.479722    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.544380 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:20.544328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03d3c4180aa732023cfc58b2079a1b5.slice/crio-1e26f5caf9d3bd665a75af3cb95392225a72b2ad71608eff80e1d8bcc6a5d370 WatchSource:0}: Error finding container 1e26f5caf9d3bd665a75af3cb95392225a72b2ad71608eff80e1d8bcc6a5d370: Status 404 returned error can't find the container with id 1e26f5caf9d3bd665a75af3cb95392225a72b2ad71608eff80e1d8bcc6a5d370
Jul 14 22:09:20.548379 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:20.548345    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55527337be11eb05926e7cca822eb38f.slice/crio-1314ce1b6dc2cb44232d96ddda68ad7d086c8f427455c3940c1a21e9e0a573d7 WatchSource:0}: Error finding container 1314ce1b6dc2cb44232d96ddda68ad7d086c8f427455c3940c1a21e9e0a573d7: Status 404 returned error can't find the container with id 1314ce1b6dc2cb44232d96ddda68ad7d086c8f427455c3940c1a21e9e0a573d7
Jul 14 22:09:20.549857 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.549837    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:09:20.580355 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.580314    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.598813 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.598762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal" event={"ID":"55527337be11eb05926e7cca822eb38f","Type":"ContainerStarted","Data":"1314ce1b6dc2cb44232d96ddda68ad7d086c8f427455c3940c1a21e9e0a573d7"}
Jul 14 22:09:20.599840 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.599819    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" event={"ID":"a03d3c4180aa732023cfc58b2079a1b5","Type":"ContainerStarted","Data":"1e26f5caf9d3bd665a75af3cb95392225a72b2ad71608eff80e1d8bcc6a5d370"}
Jul 14 22:09:20.680762 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:20.680725    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-117.ec2.internal\" not found"
Jul 14 22:09:20.745435 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.745414    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 22:09:20.752600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.752580    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:20.769076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.769048    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 22:09:20.770156 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.770137    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal"
Jul 14 22:09:20.781590 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.781569    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 22:09:20.991739 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:20.991652    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 22:09:21.435547 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.435466    2580 apiserver.go:52] "Watching apiserver"
Jul 14 22:09:21.445013 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.444984    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 22:09:21.446295 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.446258    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q","openshift-dns/node-resolver-fs8sr","openshift-image-registry/node-ca-2ncv7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal","openshift-multus/multus-wcx4n","openshift-multus/network-metrics-daemon-hwxnr","kube-system/global-pull-secret-syncer-rpzvt","kube-system/konnectivity-agent-tgbvb","openshift-cluster-node-tuning-operator/tuned-dr4rs","openshift-multus/multus-additional-cni-plugins-wkfmk","openshift-network-diagnostics/network-check-target-95v4j","openshift-network-operator/iptables-alerter-d2wbc","openshift-ovn-kubernetes/ovnkube-node-85x5g"]
Jul 14 22:09:21.447859 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.447824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:21.447980 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.447927    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:21.449278 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.449248    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.450934 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.450911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.452209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.452184    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 22:09:21.452209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.452202    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.453417 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.452995    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.453417 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.453145    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 22:09:21.453417 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.453016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 22:09:21.453417 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.453261    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 22:09:21.453675 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.453528    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.453791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.453776    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j6txh\""
Jul 14 22:09:21.453905 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.453886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.454257 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.454202    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.454756 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.454551    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vn8qq\""
Jul 14 22:09:21.454871 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.454847    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.455207 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.455189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.455771 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.455574    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 22:09:21.455771 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.455638    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lkqr6\""
Jul 14 22:09:21.455771 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.455644    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.456543 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.456513    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.458042 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.458016    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 22:09:21.458356 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.458336    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 22:09:21.458642 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.458622    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.458874    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.458902    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.459191    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2rqrb\""
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.459200    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.459434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.459504    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9f8qp\""
Jul 14 22:09:21.460153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.459886    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.461849 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.461278    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.462317 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.462295    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-sfn4f\""
Jul 14 22:09:21.462404 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.462322    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 22:09:21.462911 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.462892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 22:09:21.463324 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.463204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.464018 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.463999    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-j89dg\""
Jul 14 22:09:21.464341 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.464319    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 22:09:21.465073 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.464641    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.465073 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.464685    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.465073 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.464938    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.465428 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.465505 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465427    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 22:09:21.465619 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-system-cni-dir\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.465682 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5884e1b-5322-4f0a-8f7e-68993c54be15-host\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.465682 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465669    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-systemd-units\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.465780 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465727    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.465780 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-var-lib-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.465780 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-ovnkube-script-lib\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.465947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.465947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.465947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc47g\" (UniqueName: \"kubernetes.io/projected/b5884e1b-5322-4f0a-8f7e-68993c54be15-kube-api-access-tc47g\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.465947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.465937    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-59qb8\""
Jul 14 22:09:21.466183 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466007    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.466183 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466073    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 22:09:21.466183 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466139    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-slash\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.466350 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.466350 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-node-log\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.466350 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-log-socket\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.466508 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466375    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-run-ovn-kubernetes\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.466742 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.466828 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.466794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:21.467151 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.466417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:21.467151 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8nww\" (UniqueName: \"kubernetes.io/projected/502320b9-2d9b-411d-bc07-408eadcb47cb-kube-api-access-b8nww\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.467313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-ovnkube-config\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.467370 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d0f8817-5cd4-4623-981d-8fea3863face-ovn-node-metrics-cert\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.467425 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn7s5\" (UniqueName: \"kubernetes.io/projected/bec676e5-33e5-449e-8f6b-22080dc918ff-kube-api-access-bn7s5\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:21.467425 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cnibin\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.467524 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.467524 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-systemd\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.467743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-cni-bin\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.467743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-kubelet\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.467743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-env-overrides\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.467743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.467743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgpsb\" (UniqueName: \"kubernetes.io/projected/4191143e-ef8a-45bf-b6f2-900fcd15b83e-kube-api-access-qgpsb\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.467743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467734    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502320b9-2d9b-411d-bc07-408eadcb47cb-tmp-dir\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5884e1b-5322-4f0a-8f7e-68993c54be15-serviceca\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-os-release\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-run-netns\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-etc-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-ovn\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467897    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-cni-netd\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zd4l\" (UniqueName: \"kubernetes.io/projected/7d0f8817-5cd4-4623-981d-8fea3863face-kube-api-access-6zd4l\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.468041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.467956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502320b9-2d9b-411d-bc07-408eadcb47cb-hosts-file\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.468861 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.468050    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 22:09:21.468861 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.468366    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ggb8x\""
Jul 14 22:09:21.468861 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.468421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:21.468861 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.468486    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:21.468861 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.468784    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 22:09:21.487446 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.487419    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 22:09:21.490937 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.490915    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-42nft"
Jul 14 22:09:21.497331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.497283    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-42nft"
Jul 14 22:09:21.553845 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.553812    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 22:09:21.568214 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysctl-d\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.568393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.568393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc47g\" (UniqueName: \"kubernetes.io/projected/b5884e1b-5322-4f0a-8f7e-68993c54be15-kube-api-access-tc47g\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.568393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-slash\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-node-log\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-log-socket\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-run-ovn-kubernetes\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-k8s-cni-cncf-io\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7smtl\" (UniqueName: \"kubernetes.io/projected/fd789089-5171-4b0f-a5f8-e603dd85b945-kube-api-access-7smtl\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-slash\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568463    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-node-log\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568509    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-os-release\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-etc-selinux\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-run-ovn-kubernetes\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568564    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-var-lib-kubelet\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568568    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-log-socket\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-tmp\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/951fc85f-b636-49b9-940f-96604cd2a47e-kubelet-config\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cnibin\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.568661    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:21.568762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568667    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-cni-bin\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cnibin\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.568777    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:09:22.068728917 +0000 UTC m=+2.954574506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568802    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-cni-bin\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-netns\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568880    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysconfig\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e4282920-0536-4ca7-82b0-cf9816fbe7c3-konnectivity-ca\") pod \"konnectivity-agent-tgbvb\" (UID: \"e4282920-0536-4ca7-82b0-cf9816fbe7c3\") " pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/951fc85f-b636-49b9-940f-96604cd2a47e-dbus\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568977    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.568986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-kubelet\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-kubelet\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-etc-kubernetes\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-socket-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgpsb\" (UniqueName: \"kubernetes.io/projected/4191143e-ef8a-45bf-b6f2-900fcd15b83e-kube-api-access-qgpsb\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.569573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502320b9-2d9b-411d-bc07-408eadcb47cb-tmp-dir\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-cni-bin\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-cni-multus\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-hostroot\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e4282920-0536-4ca7-82b0-cf9816fbe7c3-agent-certs\") pod \"konnectivity-agent-tgbvb\" (UID: \"e4282920-0536-4ca7-82b0-cf9816fbe7c3\") " pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr8kv\" (UniqueName: \"kubernetes.io/projected/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-kube-api-access-mr8kv\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569426    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-os-release\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-run-netns\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-cni-netd\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-run-netns\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-lib-modules\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-os-release\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-kubelet\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-cni-netd\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502320b9-2d9b-411d-bc07-408eadcb47cb-tmp-dir\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-host-slash\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.570313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5884e1b-5322-4f0a-8f7e-68993c54be15-host\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-multus-certs\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmfrh\" (UniqueName: \"kubernetes.io/projected/0c890df1-b64e-4c63-9a21-811a4044d04c-kube-api-access-dmfrh\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5884e1b-5322-4f0a-8f7e-68993c54be15-host\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569746    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-kubernetes\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysctl-conf\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-ovnkube-config\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-cnibin\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.569915    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8nww\" (UniqueName: \"kubernetes.io/projected/502320b9-2d9b-411d-bc07-408eadcb47cb-kube-api-access-b8nww\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d0f8817-5cd4-4623-981d-8fea3863face-ovn-node-metrics-cert\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570123    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-daemon-config\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-run\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-host\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.571204 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn7s5\" (UniqueName: \"kubernetes.io/projected/bec676e5-33e5-449e-8f6b-22080dc918ff-kube-api-access-bn7s5\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-systemd\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-system-cni-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-systemd\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd789089-5171-4b0f-a5f8-e603dd85b945-cni-binary-copy\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-registration-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-systemd\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-env-overrides\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-device-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570690    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-iptables-alerter-script\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5884e1b-5322-4f0a-8f7e-68993c54be15-serviceca\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-socket-dir-parent\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-conf-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.572043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570900    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-sys\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.570960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-tuned\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-etc-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-ovn\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571124    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zd4l\" (UniqueName: \"kubernetes.io/projected/7d0f8817-5cd4-4623-981d-8fea3863face-kube-api-access-6zd4l\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502320b9-2d9b-411d-bc07-408eadcb47cb-hosts-file\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-run-ovn\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571237    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-cni-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502320b9-2d9b-411d-bc07-408eadcb47cb-hosts-file\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t447\" (UniqueName: \"kubernetes.io/projected/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-kube-api-access-5t447\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b5884e1b-5322-4f0a-8f7e-68993c54be15-serviceca\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-system-cni-dir\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571371    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-systemd-units\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4191143e-ef8a-45bf-b6f2-900fcd15b83e-system-cni-dir\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-var-lib-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-env-overrides\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.572643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571423    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-ovnkube-config\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-ovnkube-script-lib\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-etc-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-sys-fs\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-systemd-units\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571552    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d0f8817-5cd4-4623-981d-8fea3863face-var-lib-openvswitch\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571558    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-modprobe-d\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4191143e-ef8a-45bf-b6f2-900fcd15b83e-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.573293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.571908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d0f8817-5cd4-4623-981d-8fea3863face-ovnkube-script-lib\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.574483 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.574462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d0f8817-5cd4-4623-981d-8fea3863face-ovn-node-metrics-cert\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.579516 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.579490    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc47g\" (UniqueName: \"kubernetes.io/projected/b5884e1b-5322-4f0a-8f7e-68993c54be15-kube-api-access-tc47g\") pod \"node-ca-2ncv7\" (UID: \"b5884e1b-5322-4f0a-8f7e-68993c54be15\") " pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.580832 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.580781    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgpsb\" (UniqueName: \"kubernetes.io/projected/4191143e-ef8a-45bf-b6f2-900fcd15b83e-kube-api-access-qgpsb\") pod \"multus-additional-cni-plugins-wkfmk\" (UID: \"4191143e-ef8a-45bf-b6f2-900fcd15b83e\") " pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.581506 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.581485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zd4l\" (UniqueName: \"kubernetes.io/projected/7d0f8817-5cd4-4623-981d-8fea3863face-kube-api-access-6zd4l\") pod \"ovnkube-node-85x5g\" (UID: \"7d0f8817-5cd4-4623-981d-8fea3863face\") " pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.581612 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.581559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn7s5\" (UniqueName: \"kubernetes.io/projected/bec676e5-33e5-449e-8f6b-22080dc918ff-kube-api-access-bn7s5\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:21.581920 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.581899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8nww\" (UniqueName: \"kubernetes.io/projected/502320b9-2d9b-411d-bc07-408eadcb47cb-kube-api-access-b8nww\") pod \"node-resolver-fs8sr\" (UID: \"502320b9-2d9b-411d-bc07-408eadcb47cb\") " pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.671899 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.671865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-cni-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.671899 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.671905    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t447\" (UniqueName: \"kubernetes.io/projected/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-kube-api-access-5t447\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.671929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-sys-fs\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.671952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-modprobe-d\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.671978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysctl-d\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-k8s-cni-cncf-io\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7smtl\" (UniqueName: \"kubernetes.io/projected/fd789089-5171-4b0f-a5f8-e603dd85b945-kube-api-access-7smtl\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-sys-fs\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-os-release\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-etc-selinux\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-cni-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-var-lib-kubelet\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-tmp\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-os-release\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysctl-d\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/951fc85f-b636-49b9-940f-96604cd2a47e-kubelet-config\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-etc-selinux\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-netns\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672282    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-var-lib-kubelet\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/951fc85f-b636-49b9-940f-96604cd2a47e-kubelet-config\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-modprobe-d\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysconfig\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-k8s-cni-cncf-io\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-netns\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e4282920-0536-4ca7-82b0-cf9816fbe7c3-konnectivity-ca\") pod \"konnectivity-agent-tgbvb\" (UID: \"e4282920-0536-4ca7-82b0-cf9816fbe7c3\") " pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysconfig\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/951fc85f-b636-49b9-940f-96604cd2a47e-dbus\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672394    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-etc-kubernetes\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-socket-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.672801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-cni-bin\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-cni-multus\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.672525    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/951fc85f-b636-49b9-940f-96604cd2a47e-dbus\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-cni-multus\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.672601    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:09:22.172578752 +0000 UTC m=+3.058424324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-etc-kubernetes\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-hostroot\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672628    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-cni-bin\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672679    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-hostroot\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e4282920-0536-4ca7-82b0-cf9816fbe7c3-agent-certs\") pod \"konnectivity-agent-tgbvb\" (UID: \"e4282920-0536-4ca7-82b0-cf9816fbe7c3\") " pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr8kv\" (UniqueName: \"kubernetes.io/projected/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-kube-api-access-mr8kv\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672738    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-socket-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-lib-modules\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-kubelet\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-host-slash\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-multus-certs\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.673678 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmfrh\" (UniqueName: \"kubernetes.io/projected/0c890df1-b64e-4c63-9a21-811a4044d04c-kube-api-access-dmfrh\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-kubernetes\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysctl-conf\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e4282920-0536-4ca7-82b0-cf9816fbe7c3-konnectivity-ca\") pod \"konnectivity-agent-tgbvb\" (UID: \"e4282920-0536-4ca7-82b0-cf9816fbe7c3\") " pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.672985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-run-multus-certs\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-host-var-lib-kubelet\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-cnibin\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-cnibin\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673151    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-daemon-config\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-run\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-lib-modules\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-host\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-sysctl-conf\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-system-cni-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-kubernetes\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd789089-5171-4b0f-a5f8-e603dd85b945-cni-binary-copy\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-host\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.674556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-run\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-registration-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673484    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-systemd\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-device-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-system-cni-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-iptables-alerter-script\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-systemd\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-socket-dir-parent\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-registration-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-conf-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-sys\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-tuned\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd789089-5171-4b0f-a5f8-e603dd85b945-cni-binary-copy\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-device-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673920    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-conf-dir\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-socket-dir-parent\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673977    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-sys\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.675465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-host-slash\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.676130 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.673981    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0c890df1-b64e-4c63-9a21-811a4044d04c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.676130 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.674066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-iptables-alerter-script\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.676130 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.674487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fd789089-5171-4b0f-a5f8-e603dd85b945-multus-daemon-config\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.676130 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.675232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-tmp\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.676130 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.675803    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e4282920-0536-4ca7-82b0-cf9816fbe7c3-agent-certs\") pod \"konnectivity-agent-tgbvb\" (UID: \"e4282920-0536-4ca7-82b0-cf9816fbe7c3\") " pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.676471 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.676443    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-etc-tuned\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.683879 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.683849    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:21.683879 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.683876    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:21.684087 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.683892    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:21.684087 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:21.683965    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:09:22.183950397 +0000 UTC m=+3.069795951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:21.686294 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.686234    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t447\" (UniqueName: \"kubernetes.io/projected/de756ae9-f6ac-4651-9d68-0eb56a5e2bf8-kube-api-access-5t447\") pod \"tuned-dr4rs\" (UID: \"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8\") " pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:21.686808 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.686776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmfrh\" (UniqueName: \"kubernetes.io/projected/0c890df1-b64e-4c63-9a21-811a4044d04c-kube-api-access-dmfrh\") pod \"aws-ebs-csi-driver-node-rzp7q\" (UID: \"0c890df1-b64e-4c63-9a21-811a4044d04c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.687098 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.687074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7smtl\" (UniqueName: \"kubernetes.io/projected/fd789089-5171-4b0f-a5f8-e603dd85b945-kube-api-access-7smtl\") pod \"multus-wcx4n\" (UID: \"fd789089-5171-4b0f-a5f8-e603dd85b945\") " pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.687190 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.687153    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr8kv\" (UniqueName: \"kubernetes.io/projected/7680a2f5-824f-4eb3-aa2d-84f2e0f7afee-kube-api-access-mr8kv\") pod \"iptables-alerter-d2wbc\" (UID: \"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee\") " pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.767685 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.767645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wkfmk"
Jul 14 22:09:21.776432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.776394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fs8sr"
Jul 14 22:09:21.785317 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.785282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2ncv7"
Jul 14 22:09:21.791405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.791380    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:21.798340 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.798312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wcx4n"
Jul 14 22:09:21.806706 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.806677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:21.815675 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.815654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-d2wbc"
Jul 14 22:09:21.823361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.823338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q"
Jul 14 22:09:21.831005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:21.830986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dr4rs"
Jul 14 22:09:22.052742 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.052649    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 22:09:22.076391 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.076347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:22.076576 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.076537    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:22.076685 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.076671    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:09:23.076645663 +0000 UTC m=+3.962491223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:22.177776 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.177737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:22.177947 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.177922    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:22.178022 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.178010    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:09:23.177993363 +0000 UTC m=+4.063838919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:22.279076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.279021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:22.279288 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.279246    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:22.279288 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.279278    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:22.279415 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.279295    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:22.279415 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.279372    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:09:23.279353759 +0000 UTC m=+4.165199314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:22.500066 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.499956    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 22:04:21 +0000 UTC" deadline="2028-04-12 11:49:57.334556296 +0000 UTC"
Jul 14 22:09:22.500066 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.500012    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15301h40m34.834548614s"
Jul 14 22:09:22.525734 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.525702    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0f8817_5cd4_4623_981d_8fea3863face.slice/crio-10cb808cea0e300b2662b18782c795032b2682040d35c885e77a78db69986c16 WatchSource:0}: Error finding container 10cb808cea0e300b2662b18782c795032b2682040d35c885e77a78db69986c16: Status 404 returned error can't find the container with id 10cb808cea0e300b2662b18782c795032b2682040d35c885e77a78db69986c16
Jul 14 22:09:22.544573 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.544541    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4282920_0536_4ca7_82b0_cf9816fbe7c3.slice/crio-15ccd5a5f13545da172a75b08d6ef10724c0b1cdaec96e246157480d8e0984a3 WatchSource:0}: Error finding container 15ccd5a5f13545da172a75b08d6ef10724c0b1cdaec96e246157480d8e0984a3: Status 404 returned error can't find the container with id 15ccd5a5f13545da172a75b08d6ef10724c0b1cdaec96e246157480d8e0984a3
Jul 14 22:09:22.544965 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.544896    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5884e1b_5322_4f0a_8f7e_68993c54be15.slice/crio-2a824e6cad11905ea052af59f595b7b159c7c4ae42879474c4b25bdd3154a8ec WatchSource:0}: Error finding container 2a824e6cad11905ea052af59f595b7b159c7c4ae42879474c4b25bdd3154a8ec: Status 404 returned error can't find the container with id 2a824e6cad11905ea052af59f595b7b159c7c4ae42879474c4b25bdd3154a8ec
Jul 14 22:09:22.546408 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.546381    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd789089_5171_4b0f_a5f8_e603dd85b945.slice/crio-39f93be3af2d7fc7fca7b8aad5c5f4a61d72f08d49617e4998a896fd12df246d WatchSource:0}: Error finding container 39f93be3af2d7fc7fca7b8aad5c5f4a61d72f08d49617e4998a896fd12df246d: Status 404 returned error can't find the container with id 39f93be3af2d7fc7fca7b8aad5c5f4a61d72f08d49617e4998a896fd12df246d
Jul 14 22:09:22.547922 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.547890    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502320b9_2d9b_411d_bc07_408eadcb47cb.slice/crio-69d9325e3761ed637d2d02d576b5e891f7a6b12460478ca33ee9e69729f0813c WatchSource:0}: Error finding container 69d9325e3761ed637d2d02d576b5e891f7a6b12460478ca33ee9e69729f0813c: Status 404 returned error can't find the container with id 69d9325e3761ed637d2d02d576b5e891f7a6b12460478ca33ee9e69729f0813c
Jul 14 22:09:22.551409 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.548917    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4191143e_ef8a_45bf_b6f2_900fcd15b83e.slice/crio-c63f9a7fc92b807885e07a0d7f1c797aefdba238a95db5a7b0cac806846813af WatchSource:0}: Error finding container c63f9a7fc92b807885e07a0d7f1c797aefdba238a95db5a7b0cac806846813af: Status 404 returned error can't find the container with id c63f9a7fc92b807885e07a0d7f1c797aefdba238a95db5a7b0cac806846813af
Jul 14 22:09:22.551409 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.549463    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde756ae9_f6ac_4651_9d68_0eb56a5e2bf8.slice/crio-07191d49cd4c0ed23559c853a4d1ca11fd296523a9b2ef04c17c220582f3b4cf WatchSource:0}: Error finding container 07191d49cd4c0ed23559c853a4d1ca11fd296523a9b2ef04c17c220582f3b4cf: Status 404 returned error can't find the container with id 07191d49cd4c0ed23559c853a4d1ca11fd296523a9b2ef04c17c220582f3b4cf
Jul 14 22:09:22.551409 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.550609    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7680a2f5_824f_4eb3_aa2d_84f2e0f7afee.slice/crio-931c8f1b33782c6abbb3b387332b29adf7b5880e62fdc136e6df04ddf3973b92 WatchSource:0}: Error finding container 931c8f1b33782c6abbb3b387332b29adf7b5880e62fdc136e6df04ddf3973b92: Status 404 returned error can't find the container with id 931c8f1b33782c6abbb3b387332b29adf7b5880e62fdc136e6df04ddf3973b92
Jul 14 22:09:22.551409 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:09:22.550919    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c890df1_b64e_4c63_9a21_811a4044d04c.slice/crio-5677e08918334ed72fd7442163cf895fae05f61967c3b327e59def58becb3526 WatchSource:0}: Error finding container 5677e08918334ed72fd7442163cf895fae05f61967c3b327e59def58becb3526: Status 404 returned error can't find the container with id 5677e08918334ed72fd7442163cf895fae05f61967c3b327e59def58becb3526
Jul 14 22:09:22.595773 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.595593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:22.595916 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:22.595878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:22.604288 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.604250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d2wbc" event={"ID":"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee","Type":"ContainerStarted","Data":"931c8f1b33782c6abbb3b387332b29adf7b5880e62fdc136e6df04ddf3973b92"}
Jul 14 22:09:22.605587 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.605528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerStarted","Data":"c63f9a7fc92b807885e07a0d7f1c797aefdba238a95db5a7b0cac806846813af"}
Jul 14 22:09:22.606837 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.606807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fs8sr" event={"ID":"502320b9-2d9b-411d-bc07-408eadcb47cb","Type":"ContainerStarted","Data":"69d9325e3761ed637d2d02d576b5e891f7a6b12460478ca33ee9e69729f0813c"}
Jul 14 22:09:22.608164 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.608139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wcx4n" event={"ID":"fd789089-5171-4b0f-a5f8-e603dd85b945","Type":"ContainerStarted","Data":"39f93be3af2d7fc7fca7b8aad5c5f4a61d72f08d49617e4998a896fd12df246d"}
Jul 14 22:09:22.609480 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.609319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tgbvb" event={"ID":"e4282920-0536-4ca7-82b0-cf9816fbe7c3","Type":"ContainerStarted","Data":"15ccd5a5f13545da172a75b08d6ef10724c0b1cdaec96e246157480d8e0984a3"}
Jul 14 22:09:22.611312 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.611288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2ncv7" event={"ID":"b5884e1b-5322-4f0a-8f7e-68993c54be15","Type":"ContainerStarted","Data":"2a824e6cad11905ea052af59f595b7b159c7c4ae42879474c4b25bdd3154a8ec"}
Jul 14 22:09:22.612524 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.612474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dr4rs" event={"ID":"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8","Type":"ContainerStarted","Data":"07191d49cd4c0ed23559c853a4d1ca11fd296523a9b2ef04c17c220582f3b4cf"}
Jul 14 22:09:22.614447 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.614101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"10cb808cea0e300b2662b18782c795032b2682040d35c885e77a78db69986c16"}
Jul 14 22:09:22.615770 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:22.615746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q" event={"ID":"0c890df1-b64e-4c63-9a21-811a4044d04c","Type":"ContainerStarted","Data":"5677e08918334ed72fd7442163cf895fae05f61967c3b327e59def58becb3526"}
Jul 14 22:09:23.088561 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.088472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:23.088725 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.088665    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:23.088806 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.088739    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:09:25.088715404 +0000 UTC m=+5.974560978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:23.189205 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.189157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:23.189391 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.189320    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:23.189452 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.189394    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:09:25.189373747 +0000 UTC m=+6.075219327 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:23.289895 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.289855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:23.290151 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.290073    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:23.290151 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.290097    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:23.290151 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.290111    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:23.290663 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.290188    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:09:25.290169649 +0000 UTC m=+6.176015230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:23.500551 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.500418    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 22:04:21 +0000 UTC" deadline="2028-03-06 11:49:38.420186594 +0000 UTC"
Jul 14 22:09:23.500551 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.500461    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14413h40m14.919730484s"
Jul 14 22:09:23.596864 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.596305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:23.596864 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.596462    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:23.597442 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.597237    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:23.597442 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:23.597368    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:23.639373 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.639285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal" event={"ID":"55527337be11eb05926e7cca822eb38f","Type":"ContainerStarted","Data":"6928d4192033edaf100771776d5668d72e2b237c09c5e6e5ab0303f7b7650b62"}
Jul 14 22:09:23.654462 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.654414    2580 generic.go:356] "Generic (PLEG): container finished" podID="a03d3c4180aa732023cfc58b2079a1b5" containerID="a82bb1f0c16a5be3f81f989d02ad525595bfa1d4fa6b828dc537e09c2a7e9c2b" exitCode=0
Jul 14 22:09:23.654640 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.654470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" event={"ID":"a03d3c4180aa732023cfc58b2079a1b5","Type":"ContainerDied","Data":"a82bb1f0c16a5be3f81f989d02ad525595bfa1d4fa6b828dc537e09c2a7e9c2b"}
Jul 14 22:09:23.656618 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:23.656427    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-117.ec2.internal" podStartSLOduration=2.656395963 podStartE2EDuration="2.656395963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:23.65569522 +0000 UTC m=+4.541540792" watchObservedRunningTime="2026-07-14 22:09:23.656395963 +0000 UTC m=+4.542241544"
Jul 14 22:09:24.595361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:24.595320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:24.595868 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:24.595469    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:24.690699 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:24.690448    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" event={"ID":"a03d3c4180aa732023cfc58b2079a1b5","Type":"ContainerStarted","Data":"4315d0c9f33c875f06d9018ce9c758800d5d13ffae51ab42b5b3586255083ff9"}
Jul 14 22:09:24.709967 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:24.709632    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-117.ec2.internal" podStartSLOduration=3.709608581 podStartE2EDuration="3.709608581s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:24.708421388 +0000 UTC m=+5.594266996" watchObservedRunningTime="2026-07-14 22:09:24.709608581 +0000 UTC m=+5.595454161"
Jul 14 22:09:25.110004 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:25.109964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:25.110231 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.110163    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:25.110306 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.110242    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:09:29.110219806 +0000 UTC m=+9.996065364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:25.210744 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:25.210701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:25.210958 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.210938    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:25.211060 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.211012    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:09:29.210992317 +0000 UTC m=+10.096837874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:25.311980 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:25.311937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:25.312235 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.312214    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:25.312338 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.312249    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:25.312338 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.312265    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:25.312338 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.312337    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:09:29.312315696 +0000 UTC m=+10.198161269 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:25.598885 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:25.598011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:25.598885 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.598142    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:25.598885 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:25.598576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:25.598885 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:25.598690    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:26.595613 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:26.595573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:26.595785 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:26.595722    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:27.595952 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:27.595458    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:27.595952 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:27.595601    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:27.596532 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:27.596510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:27.596656 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:27.596629    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:28.595339 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:28.595302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:28.596145 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:28.595483    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:29.145975 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:29.145859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:29.146190 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.146026    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:29.146190 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.146123    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:09:37.146084479 +0000 UTC m=+18.031930056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:29.246358 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:29.246300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:29.246588 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.246488    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:29.246665 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.246613    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:09:37.246588794 +0000 UTC m=+18.132434351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:29.346950 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:29.346913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:29.347160 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.347119    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:29.347160 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.347144    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:29.347160 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.347158    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:29.347354 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.347221    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:09:37.347202435 +0000 UTC m=+18.233047993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:29.596018 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:29.595926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:29.596214 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.596041    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:29.596674 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:29.596404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:29.596674 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:29.596517    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:30.594719 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:30.594691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:30.594906 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:30.594804    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:31.595306 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:31.595265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:31.595306 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:31.595313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:31.595941 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:31.595425    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:31.595941 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:31.595833    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:32.595768 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:32.595733    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:32.596171 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:32.595874    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:33.595544 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:33.595463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:33.595711 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:33.595604    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:33.595711 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:33.595636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:33.595711 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:33.595704    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:34.595411 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:34.595376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:34.595580 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:34.595493    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:35.595232 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:35.595183    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:35.595640 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:35.595336    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:35.595640 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:35.595381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:35.595640 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:35.595578    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:36.595224 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:36.595190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:36.595427 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:36.595313    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:37.207923 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:37.207892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:37.208111 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.208050    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:37.208154 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.208131    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:09:53.208115647 +0000 UTC m=+34.093961213 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:37.308740 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:37.308641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:37.308893 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.308791    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:37.308893 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.308866    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:09:53.308845586 +0000 UTC m=+34.194691144 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:37.409668 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:37.409636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:37.409859 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.409820    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:37.409859 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.409845    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:37.409859 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.409858    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:37.410033 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.409911    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:09:53.409894692 +0000 UTC m=+34.295740246 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:37.595728 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:37.595634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:37.596149 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.595775    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:37.596149 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:37.595819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:37.596149 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:37.595961    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:38.595579 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:38.595538    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:38.595777 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:38.595686    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:39.596608 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:39.596576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:39.597068 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:39.596720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:39.597068 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:39.596793    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:39.597068 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:39.596903    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:40.595030 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:40.594996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:40.595213 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:40.595122    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:41.595886 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.595590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:41.596559 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:41.595967    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:41.596559 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.596041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:41.596559 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:41.596166    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:41.724907 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.724880    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q" event={"ID":"0c890df1-b64e-4c63-9a21-811a4044d04c","Type":"ContainerStarted","Data":"352be18498afad5d375c87ab8dc3c986b3ddf26331493b5c694784326c7e9410"}
Jul 14 22:09:41.726378 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.726343    2580 generic.go:356] "Generic (PLEG): container finished" podID="4191143e-ef8a-45bf-b6f2-900fcd15b83e" containerID="0b8ef10101830a24d2fd1cbaddc44d650616b0d083aa05f23ef1d822df8846bd" exitCode=0
Jul 14 22:09:41.726482 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.726417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerDied","Data":"0b8ef10101830a24d2fd1cbaddc44d650616b0d083aa05f23ef1d822df8846bd"}
Jul 14 22:09:41.728240 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.728209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fs8sr" event={"ID":"502320b9-2d9b-411d-bc07-408eadcb47cb","Type":"ContainerStarted","Data":"07b0106eb2aaa3c85dbd206e54d1210c035c38352d4ac199cbae5dbd32c002c5"}
Jul 14 22:09:41.729803 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.729780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wcx4n" event={"ID":"fd789089-5171-4b0f-a5f8-e603dd85b945","Type":"ContainerStarted","Data":"75c09f71f7d76f3d1c84630287b8aa1d04bf3622156f428e66551d1b11407a2f"}
Jul 14 22:09:41.731607 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.731584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tgbvb" event={"ID":"e4282920-0536-4ca7-82b0-cf9816fbe7c3","Type":"ContainerStarted","Data":"8d82b7b189cf00ef467dda698d332760e3827b53e552a79e7700959f3fdd8c95"}
Jul 14 22:09:41.733474 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.733453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2ncv7" event={"ID":"b5884e1b-5322-4f0a-8f7e-68993c54be15","Type":"ContainerStarted","Data":"24757386ecaf8862bdd493e2a07b7a68c17e4e3a38e803f8237a6b82810d70e6"}
Jul 14 22:09:41.735092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.735067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dr4rs" event={"ID":"de756ae9-f6ac-4651-9d68-0eb56a5e2bf8","Type":"ContainerStarted","Data":"1bde893a97b920bfcd758ff850fc07d5347b5980f75936cc3bdf5c4dacb33f9f"}
Jul 14 22:09:41.736477 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.736460    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:09:41.736811 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.736792    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d0f8817-5cd4-4623-981d-8fea3863face" containerID="51287f18a2798e83c8adc5fdd04d8582c8191a84e3270eeda4f51628dcb28f86" exitCode=1
Jul 14 22:09:41.736878 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.736822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerDied","Data":"51287f18a2798e83c8adc5fdd04d8582c8191a84e3270eeda4f51628dcb28f86"}
Jul 14 22:09:41.736915 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.736876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"6912a6924f190a6bd8fe36f98717a27719d526ece03779228cb3d5fb866d1d03"}
Jul 14 22:09:41.787540 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.787362    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fs8sr" podStartSLOduration=7.306683941 podStartE2EDuration="21.787345878s" totalImagesPullingTime="14.480661937s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:41.770075844 +0000 UTC m=+22.655921420" watchObservedRunningTime="2026-07-14 22:09:41.787345878 +0000 UTC m=+22.673191471"
Jul 14 22:09:41.787651 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.787576    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2ncv7" podStartSLOduration=3.001567978 podStartE2EDuration="21.787570464s" totalImagesPullingTime="18.786002486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:41.787187653 +0000 UTC m=+22.673033230" watchObservedRunningTime="2026-07-14 22:09:41.787570464 +0000 UTC m=+22.673416039"
Jul 14 22:09:41.828599 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.828501    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wcx4n" podStartSLOduration=2.98994822 podStartE2EDuration="21.828479103s" totalImagesPullingTime="18.838530883s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:41.825090068 +0000 UTC m=+22.710935644" watchObservedRunningTime="2026-07-14 22:09:41.828479103 +0000 UTC m=+22.714324682"
Jul 14 22:09:41.873196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:41.873154    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tgbvb" podStartSLOduration=11.07580234 podStartE2EDuration="21.873137769s" totalImagesPullingTime="10.797335429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:41.872908072 +0000 UTC m=+22.758753648" watchObservedRunningTime="2026-07-14 22:09:41.873137769 +0000 UTC m=+22.758983344"
Jul 14 22:09:42.144844 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.144764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:42.221947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.221907    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:42.535011 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.534974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:42.595150 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.595122    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:42.595329 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:42.595231    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:42.634911 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.634877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:42.659762 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.659739    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fs8sr_502320b9-2d9b-411d-bc07-408eadcb47cb/dns-node-resolver/0.log"
Jul 14 22:09:42.743259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.743000    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:42.743259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.743163    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:09:42.743681 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.743658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"ef8d686bfde51bdb1ffc0cd0e6b66f9d08432fc60479b86e9d7fb45d7fbb8e7d"}
Jul 14 22:09:42.743792 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.743731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"978dbaf99842ee781300f504a93623651c1e2a72dd06d71333dd8fc9e9803445"}
Jul 14 22:09:42.743792 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.743766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"9e084d567a3e1a507618d238a2c8d7f4b6579286ab56c6982a395bf9ae978bd2"}
Jul 14 22:09:42.743926 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.743828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"fec17de7ae73d60cb08ca1eee4648dd74f6eb533da20ba614900e46e737e36c4"}
Jul 14 22:09:42.874533 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:42.874494    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 22:09:43.240568 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.240488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:43.342556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.342519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:43.457115 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.457071    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2ncv7_b5884e1b-5322-4f0a-8f7e-68993c54be15/node-ca/0.log"
Jul 14 22:09:43.524764 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.524596    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T22:09:42.874521004Z","UUID":"65fd61b1-243d-427d-8948-5b149fe2adc1","Handler":null,"Name":"","Endpoint":""}
Jul 14 22:09:43.526731 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.526700    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 22:09:43.526731 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.526744    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 22:09:43.595614 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.595579    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:43.595842 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.595579    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:43.595842 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:43.595733    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:43.595842 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:43.595778    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:43.749444 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.749407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:43.750174 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.750144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q" event={"ID":"0c890df1-b64e-4c63-9a21-811a4044d04c","Type":"ContainerStarted","Data":"995e7ce3121161d34aa6a22d030b716ef42590576011b69d257b88d7fdaeab50"}
Jul 14 22:09:43.751847 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.751818    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-d2wbc" event={"ID":"7680a2f5-824f-4eb3-aa2d-84f2e0f7afee","Type":"ContainerStarted","Data":"1bdf7a9d0ddee2ec5b2ad362e96256d356bf4a0d24088f4d50fed32800d08a78"}
Jul 14 22:09:43.772436 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:43.772375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dr4rs" podStartSLOduration=4.990160482 podStartE2EDuration="23.772353626s" totalImagesPullingTime="18.782193144s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:41.899338497 +0000 UTC m=+22.785184066" watchObservedRunningTime="2026-07-14 22:09:43.772353626 +0000 UTC m=+24.658199204"
Jul 14 22:09:44.595214 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:44.595179    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:44.595410 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:44.595300    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:45.202164 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.202130    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:45.335927 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.335695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:45.435680 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.435644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:45.595284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.595253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:45.595463 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:45.595398    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:45.595463 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.595424    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:45.595573 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:45.595549    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:45.657411 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.657373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:45.737003 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.736972    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:45.737686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.737662    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:45.755307 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.755249    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-d2wbc" podStartSLOduration=6.97477742 podStartE2EDuration="25.755225748s" totalImagesPullingTime="18.780448328s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:43.772143475 +0000 UTC m=+24.657989051" watchObservedRunningTime="2026-07-14 22:09:45.755225748 +0000 UTC m=+26.641071323"
Jul 14 22:09:45.756428 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.756398    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:45.756836 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.756816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tgbvb"
Jul 14 22:09:45.757559 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.757532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:45.901980 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:45.901895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:46.595489 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.595457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:46.596019 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:46.595569    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:46.706474 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.706431    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:46.759913 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.759881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q" event={"ID":"0c890df1-b64e-4c63-9a21-811a4044d04c","Type":"ContainerStarted","Data":"0ff191002ff2c448351892f612c03db65a4bfb135f9095cb1a51c29e39daf4c4"}
Jul 14 22:09:46.761597 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.761572    2580 generic.go:356] "Generic (PLEG): container finished" podID="4191143e-ef8a-45bf-b6f2-900fcd15b83e" containerID="51a4b00c71413c8d41c7a6530387ef3f51bfe2f3c4babf61877d073a41a4c9b7" exitCode=0
Jul 14 22:09:46.761723 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.761648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerDied","Data":"51a4b00c71413c8d41c7a6530387ef3f51bfe2f3c4babf61877d073a41a4c9b7"}
Jul 14 22:09:46.764785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.764760    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:09:46.765161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.765135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"9e56de8a44eb441759b16ddac856537f9358c0b8cdf33750464dafbdd9f50db1"}
Jul 14 22:09:46.779991 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:46.779950    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rzp7q" podStartSLOduration=3.278519052 podStartE2EDuration="26.779937121s" totalImagesPullingTime="23.501418069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:46.779871031 +0000 UTC m=+27.665716606" watchObservedRunningTime="2026-07-14 22:09:46.779937121 +0000 UTC m=+27.665782697"
Jul 14 22:09:47.312492 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.312449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:47.449439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.449376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:47.549296 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.549260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:47.595371 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.595283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:47.595539 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:47.595410    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:47.595539 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.595496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:47.595948 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:47.595581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:47.705405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.705377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:47.906702 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:47.906657    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:48.007807 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.007777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:48.535846 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.535818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:48.594855 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.594824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:48.595022 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:48.594918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:48.637682 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.637650    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:48.771604 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.771416    2580 generic.go:356] "Generic (PLEG): container finished" podID="4191143e-ef8a-45bf-b6f2-900fcd15b83e" containerID="5aeb1edc24bc90615745e1ef54d1685bc4b660a2347bf1f67f2d477b2f121379" exitCode=0
Jul 14 22:09:48.771745 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.771470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerDied","Data":"5aeb1edc24bc90615745e1ef54d1685bc4b660a2347bf1f67f2d477b2f121379"}
Jul 14 22:09:48.775136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.775118    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:09:48.775502 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.775478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"590c9c1062851fd4d7f88e228d7d8428d8879ed4a6e684a6998b795279bd9c14"}
Jul 14 22:09:48.775753 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.775735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:48.775821 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.775789    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:48.775891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.775874    2580 scope.go:117] "RemoveContainer" containerID="51287f18a2798e83c8adc5fdd04d8582c8191a84e3270eeda4f51628dcb28f86"
Jul 14 22:09:48.792680 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.792661    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:48.941650 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:48.941611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:49.188860 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.188768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:49.299957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.299923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:49.360546 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.360503    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:49.503785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.503752    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:49.562900 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.562860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:49.595711 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.595677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:49.595820 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.595716    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:49.595820 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:49.595807    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:49.595943 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:49.595924    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:49.781254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.781186    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:09:49.781640 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.781530    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" event={"ID":"7d0f8817-5cd4-4623-981d-8fea3863face","Type":"ContainerStarted","Data":"a8fe0db0d6ee28dcdb67caa6e5f6d6330cb14aa13bd26652a28e13940681a779"}
Jul 14 22:09:49.781789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.781767    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:49.798268 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.798240    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:09:49.811663 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.811614    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g" podStartSLOduration=10.937106117999999 podStartE2EDuration="29.811600784s" totalImagesPullingTime="18.874494666s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:49.811456513 +0000 UTC m=+30.697302089" watchObservedRunningTime="2026-07-14 22:09:49.811600784 +0000 UTC m=+30.697446359"
Jul 14 22:09:49.953710 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:49.953667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:50.048956 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:50.048894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:50.501371 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:50.501335    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:50.595671 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:50.595634    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:50.595825 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:50.595768    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:50.695298 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:50.695271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:50.786551 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:50.786466    2580 generic.go:356] "Generic (PLEG): container finished" podID="4191143e-ef8a-45bf-b6f2-900fcd15b83e" containerID="272085150de932b9b99efcb2f5cfd75b779a7f7707cfd293ef43807b445d9071" exitCode=0
Jul 14 22:09:50.786551 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:50.786504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerDied","Data":"272085150de932b9b99efcb2f5cfd75b779a7f7707cfd293ef43807b445d9071"}
Jul 14 22:09:51.117289 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:51.117204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:51.304317 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:51.304279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:51.413466 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:51.413372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:51.595082 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:51.595046    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:51.595349 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:51.595109    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:51.595349 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:51.595215    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:51.595476 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:51.595352    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:51.905966 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:51.905925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:52.595093 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:52.595053    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:52.595309 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:52.595247    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:53.227512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:53.227478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:53.227964 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.227633    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:53.227964 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.227713    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs podName:bec676e5-33e5-449e-8f6b-22080dc918ff nodeName:}" failed. No retries permitted until 2026-07-14 22:10:25.22769278 +0000 UTC m=+66.113538350 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs") pod "network-metrics-daemon-hwxnr" (UID: "bec676e5-33e5-449e-8f6b-22080dc918ff") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 22:09:53.328887 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:53.328846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:53.329083 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.329014    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:53.329176 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.329094    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret podName:951fc85f-b636-49b9-940f-96604cd2a47e nodeName:}" failed. No retries permitted until 2026-07-14 22:10:25.329072618 +0000 UTC m=+66.214918187 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret") pod "global-pull-secret-syncer-rpzvt" (UID: "951fc85f-b636-49b9-940f-96604cd2a47e") : object "kube-system"/"original-pull-secret" not registered
Jul 14 22:09:53.429866 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:53.429830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:53.430060 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.429990    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 22:09:53.430060 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.430009    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 22:09:53.430060 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.430020    2580 projected.go:196] Error preparing data for projected volume kube-api-access-zlz85 for pod openshift-network-diagnostics/network-check-target-95v4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:53.430239 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.430075    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85 podName:63b8feb4-e720-4bd9-a31a-70fd101b1535 nodeName:}" failed. No retries permitted until 2026-07-14 22:10:25.430062272 +0000 UTC m=+66.315907825 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zlz85" (UniqueName: "kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85") pod "network-check-target-95v4j" (UID: "63b8feb4-e720-4bd9-a31a-70fd101b1535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 22:09:53.595761 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:53.595719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:53.595960 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:53.595770    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:53.595960 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.595846    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:53.596085 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:53.596002    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:54.595043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:54.595010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:54.595470 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:54.595151    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:55.595537 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:55.595499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:55.595971 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:55.595616    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:55.595971 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:55.595680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:55.595971 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:55.595790    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:55.804522 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:55.804490    2580 generic.go:356] "Generic (PLEG): container finished" podID="4191143e-ef8a-45bf-b6f2-900fcd15b83e" containerID="9d8720fcd856b711abbda8bee241bf7b6e769fc3e742589e4829a11b6634f4e3" exitCode=0
Jul 14 22:09:55.804694 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:55.804523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerDied","Data":"9d8720fcd856b711abbda8bee241bf7b6e769fc3e742589e4829a11b6634f4e3"}
Jul 14 22:09:56.595115 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:56.595057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:56.595300 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:56.595187    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:56.809548 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:56.809517    2580 generic.go:356] "Generic (PLEG): container finished" podID="4191143e-ef8a-45bf-b6f2-900fcd15b83e" containerID="becba97a11ec8e4757a5080f4e68694fe5a9a2325959e0a1eab7ec44f43115eb" exitCode=0
Jul 14 22:09:56.809921 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:56.809575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerDied","Data":"becba97a11ec8e4757a5080f4e68694fe5a9a2325959e0a1eab7ec44f43115eb"}
Jul 14 22:09:57.595311 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:57.595277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:57.595500 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:57.595274    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:57.595500 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:57.595396    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:09:57.595500 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:57.595468    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:57.815006 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:57.814976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" event={"ID":"4191143e-ef8a-45bf-b6f2-900fcd15b83e","Type":"ContainerStarted","Data":"40748b02e22a30a99038f1194da24181a568307b741a4f16ce13a6e52941e281"}
Jul 14 22:09:57.839040 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:57.838992    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wkfmk" podStartSLOduration=8.245157624 podStartE2EDuration="37.838978437s" totalImagesPullingTime="29.593820813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:09:57.837625755 +0000 UTC m=+38.723471332" watchObservedRunningTime="2026-07-14 22:09:57.838978437 +0000 UTC m=+38.724824013"
Jul 14 22:09:58.595607 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:58.595573    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:09:58.595794 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:58.595689    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:09:59.596459 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:59.596415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:09:59.596904 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:59.596518    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:09:59.596904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:09:59.596561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:09:59.596904 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:09:59.596699    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:10:00.594827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:00.594789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:00.595004 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:00.594901    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:10:01.595599 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:01.595429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:01.595952 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:01.595514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:01.595952 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:01.595662    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:10:01.595952 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:01.595755    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:10:02.595713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:02.595677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:02.596125 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:02.595781    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:10:03.595673 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:03.595640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:03.595821 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:03.595759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:03.596140 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:03.595853    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:10:03.596140 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:03.595760    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:10:04.352173 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:04.352135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rpzvt"]
Jul 14 22:10:04.352348 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:04.352261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:04.352399 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:04.352369    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:10:04.353121 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:04.353059    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hwxnr"]
Jul 14 22:10:04.353248 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:04.353207    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:04.353357 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:04.353330    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:10:04.353743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:04.353721    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-95v4j"]
Jul 14 22:10:04.353955 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:04.353935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:04.354087 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:04.354029    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:10:05.595537 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:05.595495    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:05.596207 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:05.595607    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:10:05.596207 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:05.595660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:05.596207 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:05.595748    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:10:06.595709 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:06.595675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:06.596074 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:06.595777    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:10:07.595495 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:07.595463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:07.595778 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:07.595568    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rpzvt" podUID="951fc85f-b636-49b9-940f-96604cd2a47e"
Jul 14 22:10:07.595778 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:07.595658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:07.596185 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:07.595796    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hwxnr" podUID="bec676e5-33e5-449e-8f6b-22080dc918ff"
Jul 14 22:10:08.595078 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.595040    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:08.595265 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:08.595167    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95v4j" podUID="63b8feb4-e720-4bd9-a31a-70fd101b1535"
Jul 14 22:10:08.911832 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.911749    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-117.ec2.internal" event="NodeReady"
Jul 14 22:10:08.912417 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.911901    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 22:10:08.959306 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.959273    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6pqlf"]
Jul 14 22:10:08.966167 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.966130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:08.967186 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.967158    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hpxdq"]
Jul 14 22:10:08.970021 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.969681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-29nlw\""
Jul 14 22:10:08.970021 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.969774    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 22:10:08.970021 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.969681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 22:10:08.976112 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.975586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:08.976474 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.976312    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6pqlf"]
Jul 14 22:10:08.978254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.978235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 22:10:08.978254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.978247    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z7z4x\""
Jul 14 22:10:08.978414 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.978348    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 22:10:08.978567 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.978554    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 22:10:08.979380 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.979364    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 22:10:08.982415 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.982389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hpxdq"]
Jul 14 22:10:08.983696 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.983657    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cv9rh"]
Jul 14 22:10:08.988877 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.988858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:08.992107 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.992077    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 22:10:08.992208 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.992188    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 22:10:08.992279 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.992262    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 22:10:08.992661 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.992643    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jl5db\""
Jul 14 22:10:08.996382 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:08.996364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cv9rh"]
Jul 14 22:10:09.136504 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51ef2a3c-9a27-423c-9e89-82d015e195f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.136504 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51ef2a3c-9a27-423c-9e89-82d015e195f2-crio-socket\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.136730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe4d33d-1455-4c57-b848-c4f997e252c1-config-volume\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.136730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136572    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq6cl\" (UniqueName: \"kubernetes.io/projected/ffe4d33d-1455-4c57-b848-c4f997e252c1-kube-api-access-mq6cl\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.136730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ef2a3c-9a27-423c-9e89-82d015e195f2-data-volume\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.136730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe4d33d-1455-4c57-b848-c4f997e252c1-metrics-tls\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.136730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nnn8\" (UniqueName: \"kubernetes.io/projected/51ef2a3c-9a27-423c-9e89-82d015e195f2-kube-api-access-4nnn8\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.136913 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af50125a-7868-4c53-9c40-307159cf7b1c-cert\") pod \"ingress-canary-cv9rh\" (UID: \"af50125a-7868-4c53-9c40-307159cf7b1c\") " pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.136913 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51ef2a3c-9a27-423c-9e89-82d015e195f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.136913 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbtcj\" (UniqueName: \"kubernetes.io/projected/af50125a-7868-4c53-9c40-307159cf7b1c-kube-api-access-hbtcj\") pod \"ingress-canary-cv9rh\" (UID: \"af50125a-7868-4c53-9c40-307159cf7b1c\") " pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.136913 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.136811    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffe4d33d-1455-4c57-b848-c4f997e252c1-tmp-dir\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.147080 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.147046    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"]
Jul 14 22:10:09.160163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.160140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:09.164367 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.164282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"]
Jul 14 22:10:09.165736 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.165712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xbjl7\""
Jul 14 22:10:09.165870 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.165853    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 14 22:10:09.237396 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ef2a3c-9a27-423c-9e89-82d015e195f2-data-volume\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.237396 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe4d33d-1455-4c57-b848-c4f997e252c1-metrics-tls\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.237655 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nnn8\" (UniqueName: \"kubernetes.io/projected/51ef2a3c-9a27-423c-9e89-82d015e195f2-kube-api-access-4nnn8\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.237655 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237575    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af50125a-7868-4c53-9c40-307159cf7b1c-cert\") pod \"ingress-canary-cv9rh\" (UID: \"af50125a-7868-4c53-9c40-307159cf7b1c\") " pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.237655 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51ef2a3c-9a27-423c-9e89-82d015e195f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.237655 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237640    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbtcj\" (UniqueName: \"kubernetes.io/projected/af50125a-7868-4c53-9c40-307159cf7b1c-kube-api-access-hbtcj\") pod \"ingress-canary-cv9rh\" (UID: \"af50125a-7868-4c53-9c40-307159cf7b1c\") " pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.237848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffe4d33d-1455-4c57-b848-c4f997e252c1-tmp-dir\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.237848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51ef2a3c-9a27-423c-9e89-82d015e195f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.237848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.237774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/51ef2a3c-9a27-423c-9e89-82d015e195f2-data-volume\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.238062 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffe4d33d-1455-4c57-b848-c4f997e252c1-tmp-dir\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.238139 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51ef2a3c-9a27-423c-9e89-82d015e195f2-crio-socket\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.238247 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/51ef2a3c-9a27-423c-9e89-82d015e195f2-crio-socket\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.238382 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238275    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe4d33d-1455-4c57-b848-c4f997e252c1-config-volume\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.238382 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238301    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/51ef2a3c-9a27-423c-9e89-82d015e195f2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.238382 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq6cl\" (UniqueName: \"kubernetes.io/projected/ffe4d33d-1455-4c57-b848-c4f997e252c1-kube-api-access-mq6cl\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.238697 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.238673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ffe4d33d-1455-4c57-b848-c4f997e252c1-config-volume\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.241641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.241516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ffe4d33d-1455-4c57-b848-c4f997e252c1-metrics-tls\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.241754 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.241532    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/51ef2a3c-9a27-423c-9e89-82d015e195f2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.241754 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.241576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/af50125a-7868-4c53-9c40-307159cf7b1c-cert\") pod \"ingress-canary-cv9rh\" (UID: \"af50125a-7868-4c53-9c40-307159cf7b1c\") " pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.247286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.247268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbtcj\" (UniqueName: \"kubernetes.io/projected/af50125a-7868-4c53-9c40-307159cf7b1c-kube-api-access-hbtcj\") pod \"ingress-canary-cv9rh\" (UID: \"af50125a-7868-4c53-9c40-307159cf7b1c\") " pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.247286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.247278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nnn8\" (UniqueName: \"kubernetes.io/projected/51ef2a3c-9a27-423c-9e89-82d015e195f2-kube-api-access-4nnn8\") pod \"insights-runtime-extractor-hpxdq\" (UID: \"51ef2a3c-9a27-423c-9e89-82d015e195f2\") " pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.247461 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.247410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq6cl\" (UniqueName: \"kubernetes.io/projected/ffe4d33d-1455-4c57-b848-c4f997e252c1-kube-api-access-mq6cl\") pod \"dns-default-6pqlf\" (UID: \"ffe4d33d-1455-4c57-b848-c4f997e252c1\") " pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.279272 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.279233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:09.287391 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.287367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hpxdq"
Jul 14 22:10:09.297196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.297171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cv9rh"
Jul 14 22:10:09.339347 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.339272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72b047c9-aceb-47ed-9664-06d184199dd8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xzvn8\" (UID: \"72b047c9-aceb-47ed-9664-06d184199dd8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:09.439926 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.439849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72b047c9-aceb-47ed-9664-06d184199dd8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xzvn8\" (UID: \"72b047c9-aceb-47ed-9664-06d184199dd8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:09.440362 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:09.440002    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 14 22:10:09.440362 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:09.440075    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72b047c9-aceb-47ed-9664-06d184199dd8-tls-certificates podName:72b047c9-aceb-47ed-9664-06d184199dd8 nodeName:}" failed. No retries permitted until 2026-07-14 22:10:09.940055047 +0000 UTC m=+50.825900622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/72b047c9-aceb-47ed-9664-06d184199dd8-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-xzvn8" (UID: "72b047c9-aceb-47ed-9664-06d184199dd8") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 14 22:10:09.482267 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.482234    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6pqlf"]
Jul 14 22:10:09.489568 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.489545    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cv9rh"]
Jul 14 22:10:09.492790 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:09.492764    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf50125a_7868_4c53_9c40_307159cf7b1c.slice/crio-80ccf16ab5d988145178b2a96762ee021e5deda7bd6717e84e60272fdb4566d0 WatchSource:0}: Error finding container 80ccf16ab5d988145178b2a96762ee021e5deda7bd6717e84e60272fdb4566d0: Status 404 returned error can't find the container with id 80ccf16ab5d988145178b2a96762ee021e5deda7bd6717e84e60272fdb4566d0
Jul 14 22:10:09.506261 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.506238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hpxdq"]
Jul 14 22:10:09.509443 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:09.509413    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51ef2a3c_9a27_423c_9e89_82d015e195f2.slice/crio-545edcfa72ea63b11fc7ad5c04d387eb7ff047100f1ebebd077b7982b9b6a162 WatchSource:0}: Error finding container 545edcfa72ea63b11fc7ad5c04d387eb7ff047100f1ebebd077b7982b9b6a162: Status 404 returned error can't find the container with id 545edcfa72ea63b11fc7ad5c04d387eb7ff047100f1ebebd077b7982b9b6a162
Jul 14 22:10:09.596020 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.595989    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:09.596184 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.596090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:09.599088 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.599065    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 22:10:09.599243 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.599119    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nwz7d\""
Jul 14 22:10:09.599243 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.599145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 22:10:09.843563 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.843525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpxdq" event={"ID":"51ef2a3c-9a27-423c-9e89-82d015e195f2","Type":"ContainerStarted","Data":"545edcfa72ea63b11fc7ad5c04d387eb7ff047100f1ebebd077b7982b9b6a162"}
Jul 14 22:10:09.844806 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.844768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6pqlf" event={"ID":"ffe4d33d-1455-4c57-b848-c4f997e252c1","Type":"ContainerStarted","Data":"bcb93fdb35336c8588f128d660dcec96f30c80af7d5eea9fee01b6dee8fcb6f9"}
Jul 14 22:10:09.846073 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.846045    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cv9rh" event={"ID":"af50125a-7868-4c53-9c40-307159cf7b1c","Type":"ContainerStarted","Data":"80ccf16ab5d988145178b2a96762ee021e5deda7bd6717e84e60272fdb4566d0"}
Jul 14 22:10:09.943122 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.943065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72b047c9-aceb-47ed-9664-06d184199dd8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xzvn8\" (UID: \"72b047c9-aceb-47ed-9664-06d184199dd8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:09.947114 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:09.947070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/72b047c9-aceb-47ed-9664-06d184199dd8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-xzvn8\" (UID: \"72b047c9-aceb-47ed-9664-06d184199dd8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:10.070329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.070288    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:10.248650 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.247420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"]
Jul 14 22:10:10.252582 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:10.252545    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72b047c9_aceb_47ed_9664_06d184199dd8.slice/crio-6339caee363d8b977d6273489745f9e310fc9574cb03e8fbe57409c611f56c5b WatchSource:0}: Error finding container 6339caee363d8b977d6273489745f9e310fc9574cb03e8fbe57409c611f56c5b: Status 404 returned error can't find the container with id 6339caee363d8b977d6273489745f9e310fc9574cb03e8fbe57409c611f56c5b
Jul 14 22:10:10.595626 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.595595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:10.598959 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.598936    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 22:10:10.600245 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.600024    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vcczq\""
Jul 14 22:10:10.600245 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.600077    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 22:10:10.852215 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:10.852085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8" event={"ID":"72b047c9-aceb-47ed-9664-06d184199dd8","Type":"ContainerStarted","Data":"6339caee363d8b977d6273489745f9e310fc9574cb03e8fbe57409c611f56c5b"}
Jul 14 22:10:15.867354 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.866899    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cv9rh" event={"ID":"af50125a-7868-4c53-9c40-307159cf7b1c","Type":"ContainerStarted","Data":"c22a1b15c4b507e862f22387e996acb05c04269717e3ae0a4e4373add8a957bd"}
Jul 14 22:10:15.868347 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.868324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpxdq" event={"ID":"51ef2a3c-9a27-423c-9e89-82d015e195f2","Type":"ContainerStarted","Data":"05fc9573952afd8c90d32086e6fc104b267a9a9d64fda6b6e97c4b3f2ea572b2"}
Jul 14 22:10:15.869885 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.869863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6pqlf" event={"ID":"ffe4d33d-1455-4c57-b848-c4f997e252c1","Type":"ContainerStarted","Data":"be595aa2422ab393137d5d29fea973925b0aecfc18c38d13ac7515486dafbb54"}
Jul 14 22:10:15.869947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.869906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6pqlf" event={"ID":"ffe4d33d-1455-4c57-b848-c4f997e252c1","Type":"ContainerStarted","Data":"7cd3c6594a4f73ca2b501df0e58236eeb9f28e2cbcf49094cf4132b05eee9178"}
Jul 14 22:10:15.869947 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.869935    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:15.871264 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.871244    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8" event={"ID":"72b047c9-aceb-47ed-9664-06d184199dd8","Type":"ContainerStarted","Data":"366b90a1f662c815dcbc78c51ad6c8aaffc4bb1abf83af64589921e7945c2de9"}
Jul 14 22:10:15.871355 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.871337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:15.876205 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.876175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8"
Jul 14 22:10:15.888546 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.888507    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cv9rh" podStartSLOduration=2.17283962 podStartE2EDuration="7.888492742s" totalImagesPullingTime="5.715653122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:15.88716709 +0000 UTC m=+56.773012878" watchObservedRunningTime="2026-07-14 22:10:15.888492742 +0000 UTC m=+56.774338312"
Jul 14 22:10:15.911582 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.911534    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6pqlf" podStartSLOduration=2.190845257 podStartE2EDuration="7.911517954s" totalImagesPullingTime="5.720672697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:15.910943453 +0000 UTC m=+56.796789031" watchObservedRunningTime="2026-07-14 22:10:15.911517954 +0000 UTC m=+56.797363529"
Jul 14 22:10:15.938492 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:15.938435    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-xzvn8" podStartSLOduration=1.985016968 podStartE2EDuration="6.938419902s" totalImagesPullingTime="4.953402934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:15.937207865 +0000 UTC m=+56.823053441" watchObservedRunningTime="2026-07-14 22:10:15.938419902 +0000 UTC m=+56.824265481"
Jul 14 22:10:16.226661 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.226582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-x4lkh"]
Jul 14 22:10:16.228452 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.228437    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.235244 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.235220    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 14 22:10:16.235329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.235231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 14 22:10:16.235379 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.235328    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 22:10:16.235444 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.235426    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hwfzf\""
Jul 14 22:10:16.235959 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.235944    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 22:10:16.235997 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.235976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 22:10:16.245799 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.245776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-x4lkh"]
Jul 14 22:10:16.287294 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.287257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cfaa129-e13e-41cf-b8d0-875f7226f3af-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.287480 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.287299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cfaa129-e13e-41cf-b8d0-875f7226f3af-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.287480 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.287327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cfaa129-e13e-41cf-b8d0-875f7226f3af-metrics-client-ca\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.287480 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.287376    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvvpr\" (UniqueName: \"kubernetes.io/projected/5cfaa129-e13e-41cf-b8d0-875f7226f3af-kube-api-access-lvvpr\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.388229 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.388193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cfaa129-e13e-41cf-b8d0-875f7226f3af-metrics-client-ca\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.388418 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.388312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvvpr\" (UniqueName: \"kubernetes.io/projected/5cfaa129-e13e-41cf-b8d0-875f7226f3af-kube-api-access-lvvpr\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.388418 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.388391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cfaa129-e13e-41cf-b8d0-875f7226f3af-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.388542 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.388425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cfaa129-e13e-41cf-b8d0-875f7226f3af-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.389044 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.389016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cfaa129-e13e-41cf-b8d0-875f7226f3af-metrics-client-ca\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.393005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.392975    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cfaa129-e13e-41cf-b8d0-875f7226f3af-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.393191 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.393172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cfaa129-e13e-41cf-b8d0-875f7226f3af-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.396959 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.396935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvvpr\" (UniqueName: \"kubernetes.io/projected/5cfaa129-e13e-41cf-b8d0-875f7226f3af-kube-api-access-lvvpr\") pod \"prometheus-operator-64785b6675-x4lkh\" (UID: \"5cfaa129-e13e-41cf-b8d0-875f7226f3af\") " pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.541697 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.541659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh"
Jul 14 22:10:16.684726 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.684695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-x4lkh"]
Jul 14 22:10:16.688353 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:16.688324    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfaa129_e13e_41cf_b8d0_875f7226f3af.slice/crio-0a73c19183202024a88d4a4dc6711daed7f61099dcdeea55a84db83c58324300 WatchSource:0}: Error finding container 0a73c19183202024a88d4a4dc6711daed7f61099dcdeea55a84db83c58324300: Status 404 returned error can't find the container with id 0a73c19183202024a88d4a4dc6711daed7f61099dcdeea55a84db83c58324300
Jul 14 22:10:16.876978 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.876880    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpxdq" event={"ID":"51ef2a3c-9a27-423c-9e89-82d015e195f2","Type":"ContainerStarted","Data":"83c4172ddd8d406221ac56fc2b2939ebd55340f07107813ed072d5d3808a2f7f"}
Jul 14 22:10:16.878391 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:16.878357    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh" event={"ID":"5cfaa129-e13e-41cf-b8d0-875f7226f3af","Type":"ContainerStarted","Data":"0a73c19183202024a88d4a4dc6711daed7f61099dcdeea55a84db83c58324300"}
Jul 14 22:10:18.890228 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:18.890185    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hpxdq" event={"ID":"51ef2a3c-9a27-423c-9e89-82d015e195f2","Type":"ContainerStarted","Data":"d38e29634b43d73aaf553728b324baa4fdacb3fd8b6c86716b8f4c6452420476"}
Jul 14 22:10:18.892701 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:18.892666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh" event={"ID":"5cfaa129-e13e-41cf-b8d0-875f7226f3af","Type":"ContainerStarted","Data":"fcd4984e984ce8ee237df53d090df0edf31d3a015b097f5e27932ebb7b21126d"}
Jul 14 22:10:18.892867 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:18.892727    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh" event={"ID":"5cfaa129-e13e-41cf-b8d0-875f7226f3af","Type":"ContainerStarted","Data":"f705ba88fbc57e8393892e7cf5d6cabcfd80ad39ebe46a6a38898081374b0b0e"}
Jul 14 22:10:18.914228 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:18.914169    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hpxdq" podStartSLOduration=1.8929211590000001 podStartE2EDuration="10.914146648s" totalImagesPullingTime="9.021225489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:10:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:18.91176406 +0000 UTC m=+59.797609666" watchObservedRunningTime="2026-07-14 22:10:18.914146648 +0000 UTC m=+59.799992226"
Jul 14 22:10:18.929195 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:18.929064    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-x4lkh" podStartSLOduration=0.945417147 podStartE2EDuration="2.929045048s" totalImagesPullingTime="1.983627901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:18.928881225 +0000 UTC m=+59.814726802" watchObservedRunningTime="2026-07-14 22:10:18.929045048 +0000 UTC m=+59.814890623"
Jul 14 22:10:20.019559 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.019518    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7897c69f68-gq96z"]
Jul 14 22:10:20.051186 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.051156    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7897c69f68-gq96z"]
Jul 14 22:10:20.051337 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.051284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.058677 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.058656    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 14 22:10:20.059248 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.059222    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 14 22:10:20.059625 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.059608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 14 22:10:20.059997 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.059979    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 14 22:10:20.060691 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.060675    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 14 22:10:20.060778 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.060738    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 14 22:10:20.060778 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.060744    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 14 22:10:20.060883 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.060743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bgx8h\""
Jul 14 22:10:20.116476 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.116441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-console-config\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.116686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.116485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-oauth-serving-cert\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.116686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.116523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-serving-cert\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.116686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.116544    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-oauth-config\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.116686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.116577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7vrz\" (UniqueName: \"kubernetes.io/projected/b365434d-eb87-4211-ba93-c097de1ddfe7-kube-api-access-c7vrz\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.116686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.116615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-service-ca\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.217469 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.217427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-console-config\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.217469 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.217475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-oauth-serving-cert\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.217720 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.217589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-serving-cert\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.217720 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.217621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-oauth-config\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.217720 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.217647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7vrz\" (UniqueName: \"kubernetes.io/projected/b365434d-eb87-4211-ba93-c097de1ddfe7-kube-api-access-c7vrz\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.217720 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.217674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-service-ca\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.218290 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.218267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-console-config\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.218402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.218268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-oauth-serving-cert\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.218402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.218275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-service-ca\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.220046 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.220021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-oauth-config\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.220046 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.220034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-serving-cert\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.242470 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.242441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7vrz\" (UniqueName: \"kubernetes.io/projected/b365434d-eb87-4211-ba93-c097de1ddfe7-kube-api-access-c7vrz\") pod \"console-7897c69f68-gq96z\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") " pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.360402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.360320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:20.522510 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.522477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7897c69f68-gq96z"]
Jul 14 22:10:20.525672 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:20.525649    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb365434d_eb87_4211_ba93_c097de1ddfe7.slice/crio-709f8c91494d7e413c39b4a00eb4d3750bdee99a0d67bcb3fc607f60110a5644 WatchSource:0}: Error finding container 709f8c91494d7e413c39b4a00eb4d3750bdee99a0d67bcb3fc607f60110a5644: Status 404 returned error can't find the container with id 709f8c91494d7e413c39b4a00eb4d3750bdee99a0d67bcb3fc607f60110a5644
Jul 14 22:10:20.661385 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.661311    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bnscr"]
Jul 14 22:10:20.691858 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.691832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.694801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.694777    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 22:10:20.694801 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.694790    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 22:10:20.695228 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.695214    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xrbwq\""
Jul 14 22:10:20.695326 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.695296    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 22:10:20.722668 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-textfile\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.722827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.722827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722711    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-root\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.722827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-accelerators-collector-config\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.722827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e44fcd1-2f2f-4996-a283-105ca1a9841a-metrics-client-ca\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.723034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-sys\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.723034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-wtmp\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.723034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6zlw\" (UniqueName: \"kubernetes.io/projected/9e44fcd1-2f2f-4996-a283-105ca1a9841a-kube-api-access-r6zlw\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.723034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.722933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-tls\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.823755 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-textfile\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.823946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.823946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823813    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-root\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.823946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-accelerators-collector-config\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.823946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e44fcd1-2f2f-4996-a283-105ca1a9841a-metrics-client-ca\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.823946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823917    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-root\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.823971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-sys\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824019    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-wtmp\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6zlw\" (UniqueName: \"kubernetes.io/projected/9e44fcd1-2f2f-4996-a283-105ca1a9841a-kube-api-access-r6zlw\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-sys\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-tls\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824176    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-textfile\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824209    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-wtmp\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824591 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e44fcd1-2f2f-4996-a283-105ca1a9841a-metrics-client-ca\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.824591 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.824520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-accelerators-collector-config\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.826388 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.826368    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.826454 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.826440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9e44fcd1-2f2f-4996-a283-105ca1a9841a-node-exporter-tls\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.842386 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.842365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6zlw\" (UniqueName: \"kubernetes.io/projected/9e44fcd1-2f2f-4996-a283-105ca1a9841a-kube-api-access-r6zlw\") pod \"node-exporter-bnscr\" (UID: \"9e44fcd1-2f2f-4996-a283-105ca1a9841a\") " pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:20.900457 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:20.900418    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897c69f68-gq96z" event={"ID":"b365434d-eb87-4211-ba93-c097de1ddfe7","Type":"ContainerStarted","Data":"709f8c91494d7e413c39b4a00eb4d3750bdee99a0d67bcb3fc607f60110a5644"}
Jul 14 22:10:21.001268 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.001236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bnscr"
Jul 14 22:10:21.025805 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:21.025766    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e44fcd1_2f2f_4996_a283_105ca1a9841a.slice/crio-d83399516b94b102039a5aa4223bba8126fa608eebaeb8570eee8ec0af004328 WatchSource:0}: Error finding container d83399516b94b102039a5aa4223bba8126fa608eebaeb8570eee8ec0af004328: Status 404 returned error can't find the container with id d83399516b94b102039a5aa4223bba8126fa608eebaeb8570eee8ec0af004328
Jul 14 22:10:21.774448 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.774410    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:10:21.779765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.777772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.781029 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.781003    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 14 22:10:21.781400 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.781383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 14 22:10:21.781831 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.781795    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 14 22:10:21.781931 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.781843    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 14 22:10:21.782174 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.782158    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 14 22:10:21.782394 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.782374    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sfd9r\""
Jul 14 22:10:21.782525 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.782195    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 14 22:10:21.782660 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.782645    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 14 22:10:21.782730 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.782711    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 14 22:10:21.782883 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.782867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 14 22:10:21.799093 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.799061    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:10:21.802717 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.802689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-85x5g"
Jul 14 22:10:21.835368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835068    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835831 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835831 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.835831 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.836041 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.835813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p86xs\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-kube-api-access-p86xs\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.836137 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.836058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.836137 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.836128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.836309 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.836166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.906361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.906315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnscr" event={"ID":"9e44fcd1-2f2f-4996-a283-105ca1a9841a","Type":"ContainerStarted","Data":"d83399516b94b102039a5aa4223bba8126fa608eebaeb8570eee8ec0af004328"}
Jul 14 22:10:21.937239 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937430 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937430 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937278    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937430 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:21.937356    2580 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 14 22:10:21.937430 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937641 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:21.937446    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls podName:d1b41992-d2f4-4be9-8b26-a30529320962 nodeName:}" failed. No retries permitted until 2026-07-14 22:10:22.437427054 +0000 UTC m=+63.323272612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962") : secret "alertmanager-main-tls" not found
Jul 14 22:10:21.937641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p86xs\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-kube-api-access-p86xs\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937940 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937940 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937940 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937940 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.937770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.937940 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:10:21.937840    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle podName:d1b41992-d2f4-4be9-8b26-a30529320962 nodeName:}" failed. No retries permitted until 2026-07-14 22:10:22.437823132 +0000 UTC m=+63.323668703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962") : configmap references non-existent config key: ca-bundle.crt
Jul 14 22:10:21.938651 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.938432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.938651 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.938549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.940993 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.940752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.940993 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.940951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.941582 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.941558    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-config-volume\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.941682 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.941563    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.942306 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.942287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.942659 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.942619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-config-out\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.942745 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.942655    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.943156 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.943070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-web-config\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:21.947462 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:21.947389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p86xs\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-kube-api-access-p86xs\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:22.443123 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.443027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:22.443123 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.443094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:22.444076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.444034    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:22.445857 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.445830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:22.692517 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.692477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:10:22.760052 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.760020    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"]
Jul 14 22:10:22.762766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.762743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.766153 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766131    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 14 22:10:22.766259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766137    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d4rm5\""
Jul 14 22:10:22.766319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766135    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 14 22:10:22.766380 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766348    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 14 22:10:22.766500 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 14 22:10:22.766781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766761    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 14 22:10:22.766885 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.766839    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4t0kc33tb6thl\""
Jul 14 22:10:22.788156 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.788127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"]
Jul 14 22:10:22.847626 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/841f5f5e-81a0-446e-98e0-963c2d063053-metrics-client-ca\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.847786 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.847831 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.847866 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.847909 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.847952 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-grpc-tls\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.848005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdf2\" (UniqueName: \"kubernetes.io/projected/841f5f5e-81a0-446e-98e0-963c2d063053-kube-api-access-nhdf2\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.848005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.847993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-tls\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.912003 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.911965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnscr" event={"ID":"9e44fcd1-2f2f-4996-a283-105ca1a9841a","Type":"ContainerStarted","Data":"28c4eb03ab404920f826a42a15b72c95e0e46638fd1228af0e7c434ca99c0847"}
Jul 14 22:10:22.949060 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.948978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949060 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949318 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949318 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-grpc-tls\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949318 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdf2\" (UniqueName: \"kubernetes.io/projected/841f5f5e-81a0-446e-98e0-963c2d063053-kube-api-access-nhdf2\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949318 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-tls\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949318 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/841f5f5e-81a0-446e-98e0-963c2d063053-metrics-client-ca\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.949543 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.949369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.950239 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.950197    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/841f5f5e-81a0-446e-98e0-963c2d063053-metrics-client-ca\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.952131 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.952088    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.952500 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.952475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.952614 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.952598    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.952856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.952813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-grpc-tls\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.952856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.952837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.953013 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.952943    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/841f5f5e-81a0-446e-98e0-963c2d063053-secret-thanos-querier-tls\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:22.958608 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:22.958586    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdf2\" (UniqueName: \"kubernetes.io/projected/841f5f5e-81a0-446e-98e0-963c2d063053-kube-api-access-nhdf2\") pod \"thanos-querier-5d449f5c57-j2sqf\" (UID: \"841f5f5e-81a0-446e-98e0-963c2d063053\") " pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:23.074588 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.074543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:23.399313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.397364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"]
Jul 14 22:10:23.420140 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.420117    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:10:23.422251 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:23.422226    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1b41992_d2f4_4be9_8b26_a30529320962.slice/crio-ac07fd19706b79da3fa0a4efb2967a48922905fac69310538f2ac50db8c79651 WatchSource:0}: Error finding container ac07fd19706b79da3fa0a4efb2967a48922905fac69310538f2ac50db8c79651: Status 404 returned error can't find the container with id ac07fd19706b79da3fa0a4efb2967a48922905fac69310538f2ac50db8c79651
Jul 14 22:10:23.918479 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.918399    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897c69f68-gq96z" event={"ID":"b365434d-eb87-4211-ba93-c097de1ddfe7","Type":"ContainerStarted","Data":"c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067"}
Jul 14 22:10:23.920828 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.920796    2580 generic.go:356] "Generic (PLEG): container finished" podID="9e44fcd1-2f2f-4996-a283-105ca1a9841a" containerID="28c4eb03ab404920f826a42a15b72c95e0e46638fd1228af0e7c434ca99c0847" exitCode=0
Jul 14 22:10:23.921081 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.921056    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnscr" event={"ID":"9e44fcd1-2f2f-4996-a283-105ca1a9841a","Type":"ContainerDied","Data":"28c4eb03ab404920f826a42a15b72c95e0e46638fd1228af0e7c434ca99c0847"}
Jul 14 22:10:23.922642 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.922616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"32547361d891f1a67a23834f7565270955ac0133774b9d9db26f185cb2d6de74"}
Jul 14 22:10:23.924244 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.924215    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"ac07fd19706b79da3fa0a4efb2967a48922905fac69310538f2ac50db8c79651"}
Jul 14 22:10:23.940660 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:23.940601    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7897c69f68-gq96z" podStartSLOduration=2.187224824 podStartE2EDuration="4.940580829s" totalImagesPullingTime="2.753356005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:23.938672887 +0000 UTC m=+64.824518465" watchObservedRunningTime="2026-07-14 22:10:23.940580829 +0000 UTC m=+64.826426407"
Jul 14 22:10:24.930916 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:24.930874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnscr" event={"ID":"9e44fcd1-2f2f-4996-a283-105ca1a9841a","Type":"ContainerStarted","Data":"9d4d929f86825c85e1e65bda62ac46fdc4adc46ede3957feb2d4cc5b77b297c3"}
Jul 14 22:10:24.931389 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:24.930941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bnscr" event={"ID":"9e44fcd1-2f2f-4996-a283-105ca1a9841a","Type":"ContainerStarted","Data":"9f8eacaef033eff01420aac2ea7a969a037c877ab0460e80810f464e18b8ccc7"}
Jul 14 22:10:24.932582 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:24.932553    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="dad554b8e349e88b0fef41b04da78c73e2e42571e968da2b877e1b174f02c7f2" exitCode=0
Jul 14 22:10:24.932714 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:24.932642    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"dad554b8e349e88b0fef41b04da78c73e2e42571e968da2b877e1b174f02c7f2"}
Jul 14 22:10:24.956265 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:24.956207    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bnscr" podStartSLOduration=3.761192894 podStartE2EDuration="4.956185272s" totalImagesPullingTime="1.120064011s" totalInitContainerRuntime="74.928367ms" isStatefulPod=true podCreationTimestamp="2026-07-14 22:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:24.954125776 +0000 UTC m=+65.839971372" watchObservedRunningTime="2026-07-14 22:10:24.956185272 +0000 UTC m=+65.842030851"
Jul 14 22:10:25.269823 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.269772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:25.272982 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.272956    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 22:10:25.283242 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.283208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bec676e5-33e5-449e-8f6b-22080dc918ff-metrics-certs\") pod \"network-metrics-daemon-hwxnr\" (UID: \"bec676e5-33e5-449e-8f6b-22080dc918ff\") " pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:25.370558 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.370533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:25.373743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.373723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 22:10:25.382824 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.382801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/951fc85f-b636-49b9-940f-96604cd2a47e-original-pull-secret\") pod \"global-pull-secret-syncer-rpzvt\" (UID: \"951fc85f-b636-49b9-940f-96604cd2a47e\") " pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:25.471988 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.471950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:25.475203 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.475163    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 22:10:25.485026 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.484998    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 22:10:25.496203 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.496158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlz85\" (UniqueName: \"kubernetes.io/projected/63b8feb4-e720-4bd9-a31a-70fd101b1535-kube-api-access-zlz85\") pod \"network-check-target-95v4j\" (UID: \"63b8feb4-e720-4bd9-a31a-70fd101b1535\") " pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:25.509340 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.509319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nwz7d\""
Jul 14 22:10:25.516381 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.516360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rpzvt"
Jul 14 22:10:25.517410 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.517389    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hwxnr"
Jul 14 22:10:25.611778 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.611749    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vcczq\""
Jul 14 22:10:25.620179 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.620155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:25.683384 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.682600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rpzvt"]
Jul 14 22:10:25.688006 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:25.687967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951fc85f_b636_49b9_940f_96604cd2a47e.slice/crio-b581fe56e681527129e29354c2a615f91787a8a56e63adcea749c87cfd37aa43 WatchSource:0}: Error finding container b581fe56e681527129e29354c2a615f91787a8a56e63adcea749c87cfd37aa43: Status 404 returned error can't find the container with id b581fe56e681527129e29354c2a615f91787a8a56e63adcea749c87cfd37aa43
Jul 14 22:10:25.707002 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.706928    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hwxnr"]
Jul 14 22:10:25.769557 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.769536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-95v4j"]
Jul 14 22:10:25.771811 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:10:25.771754    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b8feb4_e720_4bd9_a31a_70fd101b1535.slice/crio-ec4aa6cc823093a26bb3e5989ed6fbcaf06478dea83f558b0538baef25311e4b WatchSource:0}: Error finding container ec4aa6cc823093a26bb3e5989ed6fbcaf06478dea83f558b0538baef25311e4b: Status 404 returned error can't find the container with id ec4aa6cc823093a26bb3e5989ed6fbcaf06478dea83f558b0538baef25311e4b
Jul 14 22:10:25.880652 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.880617    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6pqlf"
Jul 14 22:10:25.940085 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.940047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"73e244c2261ef04e4f50c6c9a7a8bbf73b6f1178d855bff2718409da0b20c7d4"}
Jul 14 22:10:25.940581 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.940148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"c87feb369c25ba0fe408e7d55ea3094ae2407188cd95ae5d7c03dadea2238d8c"}
Jul 14 22:10:25.940581 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.940194    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"f5aef6711f68b89055115a60eb28f5a3160314173dc9f6b43abe6693075a754a"}
Jul 14 22:10:25.941497 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.941469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-95v4j" event={"ID":"63b8feb4-e720-4bd9-a31a-70fd101b1535","Type":"ContainerStarted","Data":"ec4aa6cc823093a26bb3e5989ed6fbcaf06478dea83f558b0538baef25311e4b"}
Jul 14 22:10:25.942912 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.942881    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hwxnr" event={"ID":"bec676e5-33e5-449e-8f6b-22080dc918ff","Type":"ContainerStarted","Data":"94baaa23f2b7f67df94cdc45325342ad48b7aa675dd29ed5ef31ee3a2a0a1bad"}
Jul 14 22:10:25.944422 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:25.944354    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rpzvt" event={"ID":"951fc85f-b636-49b9-940f-96604cd2a47e","Type":"ContainerStarted","Data":"b581fe56e681527129e29354c2a615f91787a8a56e63adcea749c87cfd37aa43"}
Jul 14 22:10:26.956425 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.956372    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:10:26.979403 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.979368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:10:26.979589 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.979551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.985616    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b8pqq\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.985875    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.986180    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.986435    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.986614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.986792    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.986973    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.987145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.987824    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-91uvfg4pfvsk6\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.988032    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.989222    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.989508    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 22:10:26.989868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.989700    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 22:10:26.990604 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.990480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 22:10:27.002213 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:26.990682    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 22:10:27.014458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.003289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 22:10:27.014458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.005215    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 22:10:27.091848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.091848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.091848 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091838    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.091978    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-config\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7dpx\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-kube-api-access-z7dpx\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.092623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.092462    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193213 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193164    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193393 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-config\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dpx\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-kube-api-access-z7dpx\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.193703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.194180 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193743    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.194180 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.194180 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.193799    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.195891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.194664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.195891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.194849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.195891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.195167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.195891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.195213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.197483 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.197459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.198807 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.198257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.199407 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.199054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.200857 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.200797    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.202913 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.202870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.204413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.203754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.204413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.203769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-config-out\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.204413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.204227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.204413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.204332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.204658 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.204590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-config\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.205087 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.205030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.205202 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.205117    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-web-config\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.205202 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.205156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.206358 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.206319    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.206476 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.206374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.207151 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.206983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7dpx\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-kube-api-access-z7dpx\") pod \"prometheus-k8s-0\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:27.321877 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:27.321829    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:30.361563 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:30.361515    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:30.362145 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:30.361651    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:30.367178 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:30.367153    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:30.988234 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:30.988205    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:10:31.083515 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.082734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:10:31.986173 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.986134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"8233504220fcf014a889682ec6adbbc8cc03f2d6504c7e7257b812de30ca417b"}
Jul 14 22:10:31.986609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.986202    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"00621bbd6a892079cab980e8a677ef21b14ff6cf71940194c98068456f296573"}
Jul 14 22:10:31.986609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.986243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"bb47379839755a9507fb08e7623fb683ca9bca5cb58c7e45fb5988c65902ec9d"}
Jul 14 22:10:31.986609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.986277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"c2e0a1b2fc5a58120fd30f575948cb729d34285d60177420a4fed74e8cdb6f6e"}
Jul 14 22:10:31.986609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.986306    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"90d27648905380efc58940a90d4507bf6b9bba8e3cde6a3527de0f38ab74b087"}
Jul 14 22:10:31.986609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.986334    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerStarted","Data":"addc00f7564ded4f4d98cc1458c55c856615d0580c102205477b4d26fcbcde79"}
Jul 14 22:10:31.987653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.987633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-95v4j" event={"ID":"63b8feb4-e720-4bd9-a31a-70fd101b1535","Type":"ContainerStarted","Data":"5a587c52951f43738f3a75060a2248f4bd498a6bed97e7d3cb7bbb07f40c156c"}
Jul 14 22:10:31.987750 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.987693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:10:31.989331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.989310    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hwxnr" event={"ID":"bec676e5-33e5-449e-8f6b-22080dc918ff","Type":"ContainerStarted","Data":"c6314f87f392c965b78630cf738cdd1574a8fb1894685664d1f9cf245d8f2bb9"}
Jul 14 22:10:31.989496 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.989350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hwxnr" event={"ID":"bec676e5-33e5-449e-8f6b-22080dc918ff","Type":"ContainerStarted","Data":"0e8501c7e24da3a2349c0e11832c6e7c242eb57270d14cd98a553a5b8f645f95"}
Jul 14 22:10:31.990703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.990683    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb" exitCode=0
Jul 14 22:10:31.990785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.990710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"}
Jul 14 22:10:31.990785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.990776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"ad676d4e9182203746f89d2e2eb944abd68f09e40d4514bf6e4ffc62f5665b92"}
Jul 14 22:10:31.992296 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.992269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rpzvt" event={"ID":"951fc85f-b636-49b9-940f-96604cd2a47e","Type":"ContainerStarted","Data":"0e6f62eb3395c464d22e8ce647894d46b30f1fff195c60b28c1cb70722381c94"}
Jul 14 22:10:31.995229 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.995209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"74b9269de713ff2ee30721c1f9a9d444b3dde65fb4dcd627d49cda405157c508"}
Jul 14 22:10:31.995326 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.995264    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"38a062907e8f622876a09e265837856506e7968cdc04e1676b816024c9263b70"}
Jul 14 22:10:31.995326 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:31.995295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" event={"ID":"841f5f5e-81a0-446e-98e0-963c2d063053","Type":"ContainerStarted","Data":"fceac4fa07eb0272eac15571e2cb7a9f4f38b2f15fd28fee89e0b352b90ae59f"}
Jul 14 22:10:32.018058 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:32.018001    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.969955797 podStartE2EDuration="11.017984015s" totalImagesPullingTime="7.028593597s" totalInitContainerRuntime="19.434621ms" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:32.015424938 +0000 UTC m=+72.901270513" watchObservedRunningTime="2026-07-14 22:10:32.017984015 +0000 UTC m=+72.903829629"
Jul 14 22:10:32.068187 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:32.068126    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hwxnr" podStartSLOduration=66.887550863 podStartE2EDuration="1m12.06809341s" totalImagesPullingTime="5.180542547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:32.066883226 +0000 UTC m=+72.952728806" watchObservedRunningTime="2026-07-14 22:10:32.06809341 +0000 UTC m=+72.953938986"
Jul 14 22:10:32.092244 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:32.092192    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf" podStartSLOduration=2.889364736 podStartE2EDuration="10.092174711s" totalImagesPullingTime="7.202809975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:32.091208642 +0000 UTC m=+72.977054219" watchObservedRunningTime="2026-07-14 22:10:32.092174711 +0000 UTC m=+72.978020287"
Jul 14 22:10:32.107359 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:32.107280    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rpzvt" podStartSLOduration=66.881549543 podStartE2EDuration="1m12.107263685s" totalImagesPullingTime="5.225714142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:32.106889084 +0000 UTC m=+72.992734662" watchObservedRunningTime="2026-07-14 22:10:32.107263685 +0000 UTC m=+72.993109262"
Jul 14 22:10:33.001114 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:33.001067    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:33.009599 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:33.008146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d449f5c57-j2sqf"
Jul 14 22:10:33.042623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:33.042569    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-95v4j" podStartSLOduration=67.917576055 podStartE2EDuration="1m13.042546854s" totalImagesPullingTime="5.124970799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:32.158720364 +0000 UTC m=+73.044565940" watchObservedRunningTime="2026-07-14 22:10:33.042546854 +0000 UTC m=+73.928392431"
Jul 14 22:10:36.017350 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.017312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"}
Jul 14 22:10:36.017814 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.017429    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"}
Jul 14 22:10:36.017814 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.017477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"}
Jul 14 22:10:36.017814 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.017525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"}
Jul 14 22:10:36.017814 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.017560    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"}
Jul 14 22:10:36.017814 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.017600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerStarted","Data":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"}
Jul 14 22:10:36.049822 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:36.049670    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.838527764 podStartE2EDuration="10.049650691s" totalImagesPullingTime="3.184219272s" totalInitContainerRuntime="26.903655ms" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:36.048125348 +0000 UTC m=+76.933970924" watchObservedRunningTime="2026-07-14 22:10:36.049650691 +0000 UTC m=+76.935496279"
Jul 14 22:10:37.322761 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:37.322723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:10:39.628811 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:10:39.628778    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7897c69f68-gq96z"]
Jul 14 22:11:03.004385 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:03.003851    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-95v4j"
Jul 14 22:11:04.649452 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:04.649207    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7897c69f68-gq96z" podUID="b365434d-eb87-4211-ba93-c097de1ddfe7" containerName="console" containerID="cri-o://c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067" gracePeriod=15
Jul 14 22:11:04.950199 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:04.950172    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:11:05.034549 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-service-ca\") pod \"b365434d-eb87-4211-ba93-c097de1ddfe7\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") "
Jul 14 22:11:05.034725 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034563    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-oauth-config\") pod \"b365434d-eb87-4211-ba93-c097de1ddfe7\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") "
Jul 14 22:11:05.034725 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034591    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-console-config\") pod \"b365434d-eb87-4211-ba93-c097de1ddfe7\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") "
Jul 14 22:11:05.034725 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034611    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-oauth-serving-cert\") pod \"b365434d-eb87-4211-ba93-c097de1ddfe7\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") "
Jul 14 22:11:05.034725 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034654    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-serving-cert\") pod \"b365434d-eb87-4211-ba93-c097de1ddfe7\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") "
Jul 14 22:11:05.034725 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7vrz\" (UniqueName: \"kubernetes.io/projected/b365434d-eb87-4211-ba93-c097de1ddfe7-kube-api-access-c7vrz\") pod \"b365434d-eb87-4211-ba93-c097de1ddfe7\" (UID: \"b365434d-eb87-4211-ba93-c097de1ddfe7\") "
Jul 14 22:11:05.035053 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.034993    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-service-ca" (OuterVolumeSpecName: "service-ca") pod "b365434d-eb87-4211-ba93-c097de1ddfe7" (UID: "b365434d-eb87-4211-ba93-c097de1ddfe7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:05.035213 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.035116    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b365434d-eb87-4211-ba93-c097de1ddfe7" (UID: "b365434d-eb87-4211-ba93-c097de1ddfe7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:05.035213 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.035201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-console-config" (OuterVolumeSpecName: "console-config") pod "b365434d-eb87-4211-ba93-c097de1ddfe7" (UID: "b365434d-eb87-4211-ba93-c097de1ddfe7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:05.037058 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.037020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b365434d-eb87-4211-ba93-c097de1ddfe7" (UID: "b365434d-eb87-4211-ba93-c097de1ddfe7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:05.037236 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.037163    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b365434d-eb87-4211-ba93-c097de1ddfe7" (UID: "b365434d-eb87-4211-ba93-c097de1ddfe7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:05.037285 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.037267    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b365434d-eb87-4211-ba93-c097de1ddfe7-kube-api-access-c7vrz" (OuterVolumeSpecName: "kube-api-access-c7vrz") pod "b365434d-eb87-4211-ba93-c097de1ddfe7" (UID: "b365434d-eb87-4211-ba93-c097de1ddfe7"). InnerVolumeSpecName "kube-api-access-c7vrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:11:05.133310 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.133275    2580 generic.go:356] "Generic (PLEG): container finished" podID="b365434d-eb87-4211-ba93-c097de1ddfe7" containerID="c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067" exitCode=0
Jul 14 22:11:05.133578 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.133321    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7897c69f68-gq96z"
Jul 14 22:11:05.133578 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.133352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897c69f68-gq96z" event={"ID":"b365434d-eb87-4211-ba93-c097de1ddfe7","Type":"ContainerDied","Data":"c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067"}
Jul 14 22:11:05.133578 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.133409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7897c69f68-gq96z" event={"ID":"b365434d-eb87-4211-ba93-c097de1ddfe7","Type":"ContainerDied","Data":"709f8c91494d7e413c39b4a00eb4d3750bdee99a0d67bcb3fc607f60110a5644"}
Jul 14 22:11:05.133578 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.133436    2580 scope.go:117] "RemoveContainer" containerID="c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067"
Jul 14 22:11:05.136325 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.136300    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-service-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:05.136426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.136328    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-oauth-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:05.136426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.136344    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-console-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:05.136426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.136358    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b365434d-eb87-4211-ba93-c097de1ddfe7-oauth-serving-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:05.136426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.136371    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b365434d-eb87-4211-ba93-c097de1ddfe7-console-serving-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:05.136426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.136384    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7vrz\" (UniqueName: \"kubernetes.io/projected/b365434d-eb87-4211-ba93-c097de1ddfe7-kube-api-access-c7vrz\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:05.142614 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.142592    2580 scope.go:117] "RemoveContainer" containerID="c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067"
Jul 14 22:11:05.142951 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:05.142931    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067\": container with ID starting with c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067 not found: ID does not exist" containerID="c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067"
Jul 14 22:11:05.143027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.142965    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067"} err="failed to get container status \"c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067\": rpc error: code = NotFound desc = could not find container \"c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067\": container with ID starting with c5b9a43bddd0c74a0a99da7f1eb292b017428f1ac17244edfb87db9d3fd76067 not found: ID does not exist"
Jul 14 22:11:05.157827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.157802    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7897c69f68-gq96z"]
Jul 14 22:11:05.161873 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.161834    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7897c69f68-gq96z"]
Jul 14 22:11:05.600225 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:05.600190    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b365434d-eb87-4211-ba93-c097de1ddfe7" path="/var/lib/kubelet/pods/b365434d-eb87-4211-ba93-c097de1ddfe7/volumes"
Jul 14 22:11:27.323052 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:27.323009    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:27.342972 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:27.342932    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:28.233868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:28.233836    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:41.065182 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065138    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:11:41.065737 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065464    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="alertmanager" containerID="cri-o://addc00f7564ded4f4d98cc1458c55c856615d0580c102205477b4d26fcbcde79" gracePeriod=120
Jul 14 22:11:41.065737 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065525    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-metric" containerID="cri-o://00621bbd6a892079cab980e8a677ef21b14ff6cf71940194c98068456f296573" gracePeriod=120
Jul 14 22:11:41.065737 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065586    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="config-reloader" containerID="cri-o://90d27648905380efc58940a90d4507bf6b9bba8e3cde6a3527de0f38ab74b087" gracePeriod=120
Jul 14 22:11:41.065737 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065628    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy" containerID="cri-o://bb47379839755a9507fb08e7623fb683ca9bca5cb58c7e45fb5988c65902ec9d" gracePeriod=120
Jul 14 22:11:41.065737 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065617    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="prom-label-proxy" containerID="cri-o://8233504220fcf014a889682ec6adbbc8cc03f2d6504c7e7257b812de30ca417b" gracePeriod=120
Jul 14 22:11:41.065737 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.065584    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-web" containerID="cri-o://c2e0a1b2fc5a58120fd30f575948cb729d34285d60177420a4fed74e8cdb6f6e" gracePeriod=120
Jul 14 22:11:41.270254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270223    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="8233504220fcf014a889682ec6adbbc8cc03f2d6504c7e7257b812de30ca417b" exitCode=0
Jul 14 22:11:41.270254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270248    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="00621bbd6a892079cab980e8a677ef21b14ff6cf71940194c98068456f296573" exitCode=0
Jul 14 22:11:41.270254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270255    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="bb47379839755a9507fb08e7623fb683ca9bca5cb58c7e45fb5988c65902ec9d" exitCode=0
Jul 14 22:11:41.270254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270261    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="90d27648905380efc58940a90d4507bf6b9bba8e3cde6a3527de0f38ab74b087" exitCode=0
Jul 14 22:11:41.270490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270268    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="addc00f7564ded4f4d98cc1458c55c856615d0580c102205477b4d26fcbcde79" exitCode=0
Jul 14 22:11:41.270490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"8233504220fcf014a889682ec6adbbc8cc03f2d6504c7e7257b812de30ca417b"}
Jul 14 22:11:41.270490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"00621bbd6a892079cab980e8a677ef21b14ff6cf71940194c98068456f296573"}
Jul 14 22:11:41.270490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270419    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"bb47379839755a9507fb08e7623fb683ca9bca5cb58c7e45fb5988c65902ec9d"}
Jul 14 22:11:41.270490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270443    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"90d27648905380efc58940a90d4507bf6b9bba8e3cde6a3527de0f38ab74b087"}
Jul 14 22:11:41.270490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:41.270466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"addc00f7564ded4f4d98cc1458c55c856615d0580c102205477b4d26fcbcde79"}
Jul 14 22:11:42.277736 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.277700    2580 generic.go:356] "Generic (PLEG): container finished" podID="d1b41992-d2f4-4be9-8b26-a30529320962" containerID="c2e0a1b2fc5a58120fd30f575948cb729d34285d60177420a4fed74e8cdb6f6e" exitCode=0
Jul 14 22:11:42.278124 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.277763    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"c2e0a1b2fc5a58120fd30f575948cb729d34285d60177420a4fed74e8cdb6f6e"}
Jul 14 22:11:42.306421 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.306396    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:42.456155 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456076    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-tls-assets\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456155 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-web\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456155 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-web-config\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456201    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-main-db\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-config-volume\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456236    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-config-out\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456253    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-cluster-tls-config\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456279    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456312    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456345    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-metrics-client-ca\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456375    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p86xs\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-kube-api-access-p86xs\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.456907 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.456448    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle\") pod \"d1b41992-d2f4-4be9-8b26-a30529320962\" (UID: \"d1b41992-d2f4-4be9-8b26-a30529320962\") "
Jul 14 22:11:42.457078 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.457051    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:42.457463 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.457434    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:42.459522 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.459270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-config-out" (OuterVolumeSpecName: "config-out") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:11:42.459663 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.459639    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.459773 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.459747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.459925 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.459898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-kube-api-access-p86xs" (OuterVolumeSpecName: "kube-api-access-p86xs") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "kube-api-access-p86xs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:11:42.460111 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.460071    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-config-volume" (OuterVolumeSpecName: "config-volume") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.460203 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.460178    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.460259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.460216    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:11:42.461608 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.461572    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:11:42.461707 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.461622    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.464073 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.464048    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.469416 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.469393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-web-config" (OuterVolumeSpecName: "web-config") pod "d1b41992-d2f4-4be9-8b26-a30529320962" (UID: "d1b41992-d2f4-4be9-8b26-a30529320962"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:42.557986 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.557943    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-main-tls\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.557986 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.557980    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-metrics-client-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.557986 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.557990    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p86xs\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-kube-api-access-p86xs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.557986 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.557999    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558009    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d1b41992-d2f4-4be9-8b26-a30529320962-tls-assets\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558019    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558028    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-web-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558037    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558047    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-alertmanager-main-db\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558055    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-config-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558063    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d1b41992-d2f4-4be9-8b26-a30529320962-config-out\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558070    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-cluster-tls-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:42.558273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:42.558079    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d1b41992-d2f4-4be9-8b26-a30529320962-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:43.285292 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.285255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d1b41992-d2f4-4be9-8b26-a30529320962","Type":"ContainerDied","Data":"ac07fd19706b79da3fa0a4efb2967a48922905fac69310538f2ac50db8c79651"}
Jul 14 22:11:43.285693 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.285307    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.285693 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.285341    2580 scope.go:117] "RemoveContainer" containerID="8233504220fcf014a889682ec6adbbc8cc03f2d6504c7e7257b812de30ca417b"
Jul 14 22:11:43.293895 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.293686    2580 scope.go:117] "RemoveContainer" containerID="00621bbd6a892079cab980e8a677ef21b14ff6cf71940194c98068456f296573"
Jul 14 22:11:43.300942 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.300922    2580 scope.go:117] "RemoveContainer" containerID="bb47379839755a9507fb08e7623fb683ca9bca5cb58c7e45fb5988c65902ec9d"
Jul 14 22:11:43.307453 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.307428    2580 scope.go:117] "RemoveContainer" containerID="c2e0a1b2fc5a58120fd30f575948cb729d34285d60177420a4fed74e8cdb6f6e"
Jul 14 22:11:43.314188 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.314171    2580 scope.go:117] "RemoveContainer" containerID="90d27648905380efc58940a90d4507bf6b9bba8e3cde6a3527de0f38ab74b087"
Jul 14 22:11:43.316410 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.316391    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:11:43.320030 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.320009    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:11:43.322314 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.322297    2580 scope.go:117] "RemoveContainer" containerID="addc00f7564ded4f4d98cc1458c55c856615d0580c102205477b4d26fcbcde79"
Jul 14 22:11:43.329320 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.329302    2580 scope.go:117] "RemoveContainer" containerID="dad554b8e349e88b0fef41b04da78c73e2e42571e968da2b877e1b174f02c7f2"
Jul 14 22:11:43.353627 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353599    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:11:43.353910 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353891    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="prom-label-proxy"
Jul 14 22:11:43.353910 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353906    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="prom-label-proxy"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353918    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353926    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353936    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-metric"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353942    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-metric"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353949    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-web"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353955    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-web"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353965    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b365434d-eb87-4211-ba93-c097de1ddfe7" containerName="console"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353973    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b365434d-eb87-4211-ba93-c097de1ddfe7" containerName="console"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353983    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="init-config-reloader"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353989    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="init-config-reloader"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.353995    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="config-reloader"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354000    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="config-reloader"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354007    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="alertmanager"
Jul 14 22:11:43.354050 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354011    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="alertmanager"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354062    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="config-reloader"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354070    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="alertmanager"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354076    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-web"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354081    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354087    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b365434d-eb87-4211-ba93-c097de1ddfe7" containerName="console"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354092    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="prom-label-proxy"
Jul 14 22:11:43.354458 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.354120    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" containerName="kube-rbac-proxy-metric"
Jul 14 22:11:43.360569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.360542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.363628 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363597    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 14 22:11:43.363628 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 14 22:11:43.363836 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363703    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 14 22:11:43.363836 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363605    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 14 22:11:43.363951 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363927    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sfd9r\""
Jul 14 22:11:43.363951 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363933    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 14 22:11:43.364047 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363958    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 14 22:11:43.364047 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.363968    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 14 22:11:43.364390 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.364374    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 14 22:11:43.368389 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.368369    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 14 22:11:43.380816 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.376690    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:11:43.464765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464663    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-web-config\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.464765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-config-volume\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.464765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0604c386-1c02-4de1-8b05-6d46ea81eb87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.464765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5zh7\" (UniqueName: \"kubernetes.io/projected/0604c386-1c02-4de1-8b05-6d46ea81eb87-kube-api-access-h5zh7\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0604c386-1c02-4de1-8b05-6d46ea81eb87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0604c386-1c02-4de1-8b05-6d46ea81eb87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.464947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0604c386-1c02-4de1-8b05-6d46ea81eb87-config-out\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.465009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.465071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0604c386-1c02-4de1-8b05-6d46ea81eb87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.465378 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.465132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566047 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.565992    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566047 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0604c386-1c02-4de1-8b05-6d46ea81eb87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0604c386-1c02-4de1-8b05-6d46ea81eb87-config-out\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0604c386-1c02-4de1-8b05-6d46ea81eb87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-web-config\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-config-volume\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0604c386-1c02-4de1-8b05-6d46ea81eb87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5zh7\" (UniqueName: \"kubernetes.io/projected/0604c386-1c02-4de1-8b05-6d46ea81eb87-kube-api-access-h5zh7\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.566600 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.566359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0604c386-1c02-4de1-8b05-6d46ea81eb87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.567120 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.567073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0604c386-1c02-4de1-8b05-6d46ea81eb87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.567743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.567392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0604c386-1c02-4de1-8b05-6d46ea81eb87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569360 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569455 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0604c386-1c02-4de1-8b05-6d46ea81eb87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569452    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569620 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569600    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569722 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-web-config\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569764 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0604c386-1c02-4de1-8b05-6d46ea81eb87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.569857 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.569839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0604c386-1c02-4de1-8b05-6d46ea81eb87-config-out\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.570033 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.570013    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.570241 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.570224    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-config-volume\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.571392 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.571373    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0604c386-1c02-4de1-8b05-6d46ea81eb87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.581936 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.581913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5zh7\" (UniqueName: \"kubernetes.io/projected/0604c386-1c02-4de1-8b05-6d46ea81eb87-kube-api-access-h5zh7\") pod \"alertmanager-main-0\" (UID: \"0604c386-1c02-4de1-8b05-6d46ea81eb87\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.599865 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.599839    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1b41992-d2f4-4be9-8b26-a30529320962" path="/var/lib/kubelet/pods/d1b41992-d2f4-4be9-8b26-a30529320962/volumes"
Jul 14 22:11:43.669764 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.669727    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 14 22:11:43.807301 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:43.807273    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 14 22:11:43.808883 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:11:43.808856    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0604c386_1c02_4de1_8b05_6d46ea81eb87.slice/crio-17eb3f5f9c5c359f03c65dc50e6e47d491d37528bcba5371371ea1b3cd2f1464 WatchSource:0}: Error finding container 17eb3f5f9c5c359f03c65dc50e6e47d491d37528bcba5371371ea1b3cd2f1464: Status 404 returned error can't find the container with id 17eb3f5f9c5c359f03c65dc50e6e47d491d37528bcba5371371ea1b3cd2f1464
Jul 14 22:11:44.291965 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:44.291926    2580 generic.go:356] "Generic (PLEG): container finished" podID="0604c386-1c02-4de1-8b05-6d46ea81eb87" containerID="0d92a658889f006e067875f3e93cf37ac437bcce76f21e5256f415f729e08afb" exitCode=0
Jul 14 22:11:44.292356 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:44.292010    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerDied","Data":"0d92a658889f006e067875f3e93cf37ac437bcce76f21e5256f415f729e08afb"}
Jul 14 22:11:44.292356 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:44.292082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"17eb3f5f9c5c359f03c65dc50e6e47d491d37528bcba5371371ea1b3cd2f1464"}
Jul 14 22:11:45.186070 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.186022    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-d85979485-bwvm9"]
Jul 14 22:11:45.189427 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.189406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.192359 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.192332    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 14 22:11:45.192484 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.192338    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 14 22:11:45.192539 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.192521    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 14 22:11:45.192861 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.192847    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mg64b\""
Jul 14 22:11:45.193414 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.193394    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 14 22:11:45.193414 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.193406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 14 22:11:45.205629 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.202534    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 14 22:11:45.210142 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.210109    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d85979485-bwvm9"]
Jul 14 22:11:45.301033 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.301000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"4b748c88d6ad96dd78b966105d0c83a83abd56d7df0ca37e93a9192829d742d2"}
Jul 14 22:11:45.301432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.301065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"9e5d1043cf669a05cda05311767e8cb5245782c5050712c1651369f520ff8959"}
Jul 14 22:11:45.301432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.301091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"9475ff7a7e578d45de63c35b3edba9d2b746b275988a4bd2d9b22f441278bd82"}
Jul 14 22:11:45.301432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.301135    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"add754f14e27c75222793c5308e7c18ed49a9e43d5e290dc5aaa51b9de91aeb9"}
Jul 14 22:11:45.301432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.301163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"7ccfcac1c7838e82f5684c257ccb5b42c32f1c01bc80dea678940f3b7292fec2"}
Jul 14 22:11:45.301432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.301186    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0604c386-1c02-4de1-8b05-6d46ea81eb87","Type":"ContainerStarted","Data":"c38b6eb7307564ef97f7fadaa6b939512110eba72108e495e6523a8d2b925c46"}
Jul 14 22:11:45.330606 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.330543    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.314882378 podStartE2EDuration="2.330517221s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.634843ms" isStatefulPod=false podCreationTimestamp="2026-07-14 22:11:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:11:45.327502435 +0000 UTC m=+146.213348010" watchObservedRunningTime="2026-07-14 22:11:45.330517221 +0000 UTC m=+146.216362801"
Jul 14 22:11:45.382422 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fscmm\" (UniqueName: \"kubernetes.io/projected/752b042d-4925-4762-91f4-1ca883df869d-kube-api-access-fscmm\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382606 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382606 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-federate-client-tls\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382606 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-secret-telemeter-client\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-metrics-client-ca\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-telemeter-client-tls\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.382855 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.382787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-serving-certs-ca-bundle\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.477705 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.477614    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:11:45.478010 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.477979    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="prometheus" containerID="cri-o://4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f" gracePeriod=600
Jul 14 22:11:45.479788 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.478166    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy" containerID="cri-o://c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43" gracePeriod=600
Jul 14 22:11:45.479788 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.478223    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="thanos-sidecar" containerID="cri-o://b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2" gracePeriod=600
Jul 14 22:11:45.479788 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.478199    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-web" containerID="cri-o://e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d" gracePeriod=600
Jul 14 22:11:45.479788 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.478312    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="config-reloader" containerID="cri-o://9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30" gracePeriod=600
Jul 14 22:11:45.479788 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.478418    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656" gracePeriod=600
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-telemeter-client-tls\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-serving-certs-ca-bundle\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fscmm\" (UniqueName: \"kubernetes.io/projected/752b042d-4925-4762-91f4-1ca883df869d-kube-api-access-fscmm\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485716    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485918    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-federate-client-tls\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.485998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-secret-telemeter-client\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486136 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.486022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-metrics-client-ca\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.486822 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.486708    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-metrics-client-ca\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.487844 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.487401    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-serving-certs-ca-bundle\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.488468 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.488449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/752b042d-4925-4762-91f4-1ca883df869d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.494780 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.494257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-secret-telemeter-client\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.494780 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.494735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-telemeter-client-tls\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.495799 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.495759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.495976 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.495951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/752b042d-4925-4762-91f4-1ca883df869d-federate-client-tls\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.497070 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.496958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fscmm\" (UniqueName: \"kubernetes.io/projected/752b042d-4925-4762-91f4-1ca883df869d-kube-api-access-fscmm\") pod \"telemeter-client-d85979485-bwvm9\" (UID: \"752b042d-4925-4762-91f4-1ca883df869d\") " pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.500306 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.500283    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9"
Jul 14 22:11:45.691355 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.691315    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d85979485-bwvm9"]
Jul 14 22:11:45.693544 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:11:45.693504    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752b042d_4925_4762_91f4_1ca883df869d.slice/crio-370eeecd184dd47900f16cd289892011d00ac4bf6a0419ced470556778271ff8 WatchSource:0}: Error finding container 370eeecd184dd47900f16cd289892011d00ac4bf6a0419ced470556778271ff8: Status 404 returned error can't find the container with id 370eeecd184dd47900f16cd289892011d00ac4bf6a0419ced470556778271ff8
Jul 14 22:11:45.749794 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.749761    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:45.890663 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.890615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-db\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.890663 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.890668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-config\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.890925 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.890695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-1\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.890925 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.890720    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.890925 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.890761    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-0\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.890925 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.890786    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-web-config\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891024    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-tls-assets\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891092    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-serving-certs-ca-bundle\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891147    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-kube-rbac-proxy\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891197    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-config-out\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891224    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-tls\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891258    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-kubelet-serving-ca-bundle\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891324    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-metrics-client-ca\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-grpc-tls\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891409    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-metrics-client-certs\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.891529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891476    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-2\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.892031 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891041    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.892031 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.892031 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.891897    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.892265 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.892062    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7dpx\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-kube-api-access-z7dpx\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.892329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.892302    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.892380 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.892368    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:11:45.892470 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.892450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-thanos-prometheus-http-client-file\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.892556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.892520    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-trusted-ca-bundle\") pod \"8df71539-1c1c-45b1-82b3-5881e14b864a\" (UID: \"8df71539-1c1c-45b1-82b3-5881e14b864a\") "
Jul 14 22:11:45.893019 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.892989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.893139 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.893205 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893144    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893205 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893166    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893205 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893183    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893205 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893199    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893387 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893215    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-configmap-metrics-client-ca\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893387 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893231    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893387 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893248    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-k8s-db\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.893758 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:11:45.894186 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.893951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:11:45.894186 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.894138    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-config" (OuterVolumeSpecName: "config") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.894375 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.894336    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.895349 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.895313    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.895498 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.895468    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.895805 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.895783    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.895940 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.895919    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.896212 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.896194    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-kube-api-access-z7dpx" (OuterVolumeSpecName: "kube-api-access-z7dpx") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "kube-api-access-z7dpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:11:45.896358 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.896334    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.896488 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.896470    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-config-out" (OuterVolumeSpecName: "config-out") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:11:45.896524 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.896472    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.904983 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.904956    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-web-config" (OuterVolumeSpecName: "web-config") pod "8df71539-1c1c-45b1-82b3-5881e14b864a" (UID: "8df71539-1c1c-45b1-82b3-5881e14b864a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993670    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993716    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993725    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-web-config\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993734    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-tls-assets\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993743    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-kube-rbac-proxy\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993750    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8df71539-1c1c-45b1-82b3-5881e14b864a-config-out\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993760    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993768    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-grpc-tls\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993778    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-metrics-client-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.993769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993786    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.994196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993795    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7dpx\" (UniqueName: \"kubernetes.io/projected/8df71539-1c1c-45b1-82b3-5881e14b864a-kube-api-access-z7dpx\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.994196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993803    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8df71539-1c1c-45b1-82b3-5881e14b864a-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:45.994196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:45.993813    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df71539-1c1c-45b1-82b3-5881e14b864a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:11:46.305461 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.305428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9" event={"ID":"752b042d-4925-4762-91f4-1ca883df869d","Type":"ContainerStarted","Data":"370eeecd184dd47900f16cd289892011d00ac4bf6a0419ced470556778271ff8"}
Jul 14 22:11:46.308386 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308361    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656" exitCode=0
Jul 14 22:11:46.308386 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308383    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43" exitCode=0
Jul 14 22:11:46.308386 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308389    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d" exitCode=0
Jul 14 22:11:46.308569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308394    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2" exitCode=0
Jul 14 22:11:46.308569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308401    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30" exitCode=0
Jul 14 22:11:46.308569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308407    2580 generic.go:356] "Generic (PLEG): container finished" podID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f" exitCode=0
Jul 14 22:11:46.308569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308442    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.308569 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"}
Jul 14 22:11:46.308715 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"}
Jul 14 22:11:46.308715 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"}
Jul 14 22:11:46.308715 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"}
Jul 14 22:11:46.308715 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308649    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.308715 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"}
Jul 14 22:11:46.308946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308716    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"}
Jul 14 22:11:46.308946 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.308756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8df71539-1c1c-45b1-82b3-5881e14b864a","Type":"ContainerDied","Data":"ad676d4e9182203746f89d2e2eb944abd68f09e40d4514bf6e4ffc62f5665b92"}
Jul 14 22:11:46.319895 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.319878    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.326738 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.326714    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.333367 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.333344    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.337369 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.337341    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:11:46.341657 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.341639    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.342630 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.342552    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:11:46.348373 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.348358    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.355610 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.355596    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.362560 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.362543    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.362817 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.362797    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.362875 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.362826    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"} err="failed to get container status \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist"
Jul 14 22:11:46.362875 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.362846    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.363092 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.363077    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.363148 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363115    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"} err="failed to get container status \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist"
Jul 14 22:11:46.363148 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363135    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.363373 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.363355    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.363425 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363379    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"} err="failed to get container status \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist"
Jul 14 22:11:46.363425 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363396    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.363634 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.363613    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.363727 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363635    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"} err="failed to get container status \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist"
Jul 14 22:11:46.363727 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363654    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.363895 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.363879    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.363941 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363899    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"} err="failed to get container status \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist"
Jul 14 22:11:46.363941 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.363912    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.364199 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.364179    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.364254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364205    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"} err="failed to get container status \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist"
Jul 14 22:11:46.364254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364219    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.364467 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:11:46.364442    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.364566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364471    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"} err="failed to get container status \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist"
Jul 14 22:11:46.364566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364490    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.364714 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364698    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"} err="failed to get container status \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist"
Jul 14 22:11:46.364752 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364714    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.364919 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364900    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"} err="failed to get container status \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist"
Jul 14 22:11:46.364961 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.364920    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.365160 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365142    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"} err="failed to get container status \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist"
Jul 14 22:11:46.365208 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365160    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.365344 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365328    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"} err="failed to get container status \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist"
Jul 14 22:11:46.365386 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365344    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.365575 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365559    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"} err="failed to get container status \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist"
Jul 14 22:11:46.365617 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365575    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.365778 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365759    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"} err="failed to get container status \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist"
Jul 14 22:11:46.365819 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365780    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.365971 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365952    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"} err="failed to get container status \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist"
Jul 14 22:11:46.366023 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.365972    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.366213 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366198    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"} err="failed to get container status \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist"
Jul 14 22:11:46.366261 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366213    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.366422 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366403    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"} err="failed to get container status \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist"
Jul 14 22:11:46.366470 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366423    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.366611 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366595    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"} err="failed to get container status \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist"
Jul 14 22:11:46.366611 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366610    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.366790 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366774    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"} err="failed to get container status \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist"
Jul 14 22:11:46.366832 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.366791    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.367022 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367004    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"} err="failed to get container status \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist"
Jul 14 22:11:46.367068 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367023    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.367249 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367232    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"} err="failed to get container status \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist"
Jul 14 22:11:46.367249 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367248    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.367448 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367433    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"} err="failed to get container status \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist"
Jul 14 22:11:46.367448 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367448    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.367643 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367626    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"} err="failed to get container status \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist"
Jul 14 22:11:46.367686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367644    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.367852 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367834    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"} err="failed to get container status \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist"
Jul 14 22:11:46.367922 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.367855    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.368127 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368092    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"} err="failed to get container status \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist"
Jul 14 22:11:46.368182 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368128    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.368319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368303    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"} err="failed to get container status \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist"
Jul 14 22:11:46.368319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368318    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.368515 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368498    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"} err="failed to get container status \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist"
Jul 14 22:11:46.368557 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368516    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.368732 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368716    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"} err="failed to get container status \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist"
Jul 14 22:11:46.368732 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368732    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.368948 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368930    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"} err="failed to get container status \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist"
Jul 14 22:11:46.369002 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.368948    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.369191 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369176    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"} err="failed to get container status \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist"
Jul 14 22:11:46.369191 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369189    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.369387 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369369    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"} err="failed to get container status \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist"
Jul 14 22:11:46.369432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369388    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.369605 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369589    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"} err="failed to get container status \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist"
Jul 14 22:11:46.369642 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369605    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.369772 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369758    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"} err="failed to get container status \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist"
Jul 14 22:11:46.369811 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369772    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.369989 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369972    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"} err="failed to get container status \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist"
Jul 14 22:11:46.370048 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.369992    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.370222 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370204    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"} err="failed to get container status \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist"
Jul 14 22:11:46.370268 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370222    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.370443 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370427    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"} err="failed to get container status \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist"
Jul 14 22:11:46.370443 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370443    2580 scope.go:117] "RemoveContainer" containerID="5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"
Jul 14 22:11:46.370615 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370600    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656"} err="failed to get container status \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": rpc error: code = NotFound desc = could not find container \"5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656\": container with ID starting with 5d1328eee311730a1c03b2eb363c39f27ccf513caa75a47db9897acd5d6ac656 not found: ID does not exist"
Jul 14 22:11:46.370615 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370614    2580 scope.go:117] "RemoveContainer" containerID="c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"
Jul 14 22:11:46.370783 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370767    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43"} err="failed to get container status \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": rpc error: code = NotFound desc = could not find container \"c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43\": container with ID starting with c0aef6e1cb63653279c2f7e76a8c34cc4a25bb71db27ba468178834344cdaf43 not found: ID does not exist"
Jul 14 22:11:46.370828 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370783    2580 scope.go:117] "RemoveContainer" containerID="e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"
Jul 14 22:11:46.370963 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370942    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d"} err="failed to get container status \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": rpc error: code = NotFound desc = could not find container \"e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d\": container with ID starting with e6ab7b830c8bdabea4358b61f19af8db911199482bdda2f5d38f24c3b2dee16d not found: ID does not exist"
Jul 14 22:11:46.371005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.370964    2580 scope.go:117] "RemoveContainer" containerID="b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"
Jul 14 22:11:46.371161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371147    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2"} err="failed to get container status \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": rpc error: code = NotFound desc = could not find container \"b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2\": container with ID starting with b9556ec70cfaabbfacf5b5cab5c74512d355baae9603fd79f7f563532bf583a2 not found: ID does not exist"
Jul 14 22:11:46.371161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371161    2580 scope.go:117] "RemoveContainer" containerID="9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"
Jul 14 22:11:46.371336 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371321    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30"} err="failed to get container status \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": rpc error: code = NotFound desc = could not find container \"9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30\": container with ID starting with 9228754c124fe02d2bfebea92908c5f15b8cab0c30559eba46ef72796769cf30 not found: ID does not exist"
Jul 14 22:11:46.371374 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371336    2580 scope.go:117] "RemoveContainer" containerID="4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"
Jul 14 22:11:46.371531 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371515    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f"} err="failed to get container status \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": rpc error: code = NotFound desc = could not find container \"4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f\": container with ID starting with 4039a16bed7a7b196dfb018a15bff01c838e09d4792f3b858ca09dbcf0c9bd9f not found: ID does not exist"
Jul 14 22:11:46.371577 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371531    2580 scope.go:117] "RemoveContainer" containerID="f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"
Jul 14 22:11:46.371739 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.371723    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb"} err="failed to get container status \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": rpc error: code = NotFound desc = could not find container \"f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb\": container with ID starting with f1bfc1eebd4766847defe2859bdea40381d8ddc8903d6e5722534c92a0954ffb not found: ID does not exist"
Jul 14 22:11:46.374809 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.374788    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:11:46.375053 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375041    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy"
Jul 14 22:11:46.375092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375054    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy"
Jul 14 22:11:46.375092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375068    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="thanos-sidecar"
Jul 14 22:11:46.375092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375074    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="thanos-sidecar"
Jul 14 22:11:46.375092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375080    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-thanos"
Jul 14 22:11:46.375092 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375085    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-thanos"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375094    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-web"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375117    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-web"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375126    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="prometheus"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375131    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="prometheus"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375138    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="init-config-reloader"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375143    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="init-config-reloader"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375155    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="config-reloader"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375159    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="config-reloader"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375196    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="prometheus"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375203    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-thanos"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375212    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="config-reloader"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375217    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375224    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="thanos-sidecar"
Jul 14 22:11:46.375321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.375229    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" containerName="kube-rbac-proxy-web"
Jul 14 22:11:46.378499 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.378483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.381443 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.381411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b8pqq\""
Jul 14 22:11:46.381593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.381506    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 22:11:46.381593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.381523    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-91uvfg4pfvsk6\""
Jul 14 22:11:46.381933 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.381914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 22:11:46.382206 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.382063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 22:11:46.382316 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.382210    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 22:11:46.382316 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.382251    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 22:11:46.382962 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.382872    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 22:11:46.382962 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.382872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 22:11:46.383254 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.383142    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 22:11:46.387161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.387089    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 22:11:46.387810 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.387722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 22:11:46.388248 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.388114    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 22:11:46.388248 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.388212    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 22:11:46.388654 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.388538    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 22:11:46.392659 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.392639    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 22:11:46.395599 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.395580    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 22:11:46.399671 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.399671 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.399958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84de2809-7e09-4f7d-819d-743129c347a7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.399958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.399958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399791    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.399958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.399958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.399869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.400529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.400372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.400712 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.400632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-web-config\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.400712 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.400666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401007 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.400979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401208 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76xqx\" (UniqueName: \"kubernetes.io/projected/84de2809-7e09-4f7d-819d-743129c347a7-kube-api-access-76xqx\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401294 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401294 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401243    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-config\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84de2809-7e09-4f7d-819d-743129c347a7-config-out\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401413 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401439    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401478    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.401562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:11:46.401562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.401506    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.502931 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.502894    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.502931 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.502934    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.502955    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.502979    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-web-config\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76xqx\" (UniqueName: \"kubernetes.io/projected/84de2809-7e09-4f7d-819d-743129c347a7-kube-api-access-76xqx\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-config\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84de2809-7e09-4f7d-819d-743129c347a7-config-out\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503341    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84de2809-7e09-4f7d-819d-743129c347a7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.503639 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.504069 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.503774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.504163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.504137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.504290 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.504262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.504362 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.504269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.505684 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.505299    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.505684 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.505425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.505840 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.505706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.506519 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.506465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-web-config\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.506935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.506908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.507014 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.506961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/84de2809-7e09-4f7d-819d-743129c347a7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.507658 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.507599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.507910 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.507888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.508015 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.507985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.508490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.508466    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-config\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.508748 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.508728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/84de2809-7e09-4f7d-819d-743129c347a7-config-out\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.508921 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.508898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.508996 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.508980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.509248 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.509229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/84de2809-7e09-4f7d-819d-743129c347a7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.509343 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.509328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/84de2809-7e09-4f7d-819d-743129c347a7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.513818 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.513792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76xqx\" (UniqueName: \"kubernetes.io/projected/84de2809-7e09-4f7d-819d-743129c347a7-kube-api-access-76xqx\") pod \"prometheus-k8s-0\" (UID: \"84de2809-7e09-4f7d-819d-743129c347a7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.693173 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.693052    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:11:46.857628 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:46.857587    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 22:11:46.860716 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:11:46.860686    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84de2809_7e09_4f7d_819d_743129c347a7.slice/crio-0f71b13b0903bd2c0b694a41158ddb789f25ed1f5c2f44a11ae89176342a416f WatchSource:0}: Error finding container 0f71b13b0903bd2c0b694a41158ddb789f25ed1f5c2f44a11ae89176342a416f: Status 404 returned error can't find the container with id 0f71b13b0903bd2c0b694a41158ddb789f25ed1f5c2f44a11ae89176342a416f
Jul 14 22:11:47.316649 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:47.316613    2580 generic.go:356] "Generic (PLEG): container finished" podID="84de2809-7e09-4f7d-819d-743129c347a7" containerID="e60875ecb8d6c15ab3fe04de648ae818dfdce81c68e15b07bf786f681383b24d" exitCode=0
Jul 14 22:11:47.317118 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:47.316685    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerDied","Data":"e60875ecb8d6c15ab3fe04de648ae818dfdce81c68e15b07bf786f681383b24d"}
Jul 14 22:11:47.317118 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:47.316753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"0f71b13b0903bd2c0b694a41158ddb789f25ed1f5c2f44a11ae89176342a416f"}
Jul 14 22:11:47.601486 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:47.601386    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8df71539-1c1c-45b1-82b3-5881e14b864a" path="/var/lib/kubelet/pods/8df71539-1c1c-45b1-82b3-5881e14b864a/volumes"
Jul 14 22:11:48.322758 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.322722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9" event={"ID":"752b042d-4925-4762-91f4-1ca883df869d","Type":"ContainerStarted","Data":"e2048a5189d3b79e1c8ed4fa9c7cf9c2ce055879588fa2c696b65445d588cec7"}
Jul 14 22:11:48.323162 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.322829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9" event={"ID":"752b042d-4925-4762-91f4-1ca883df869d","Type":"ContainerStarted","Data":"5a8f0c5c24d000e85504f05fd408707445e3cec28f3a35a9ac092aa00b43f4b5"}
Jul 14 22:11:48.323162 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.322857    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9" event={"ID":"752b042d-4925-4762-91f4-1ca883df869d","Type":"ContainerStarted","Data":"4b2b1e9aba6d728e0c0cee75aeaefe6773897e3e024a076532acdee8ef549f46"}
Jul 14 22:11:48.326070 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.326041    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"7fa2c946c68da06b46921cd619084f20d44b070ba8a5aaabca8cd6a699af6617"}
Jul 14 22:11:48.326251 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.326114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"bb045868b9a60e29717d125f7bbb8c9260353591724046a62ba72c4ff8df0a5c"}
Jul 14 22:11:48.326251 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.326143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"8cfe9fc4f3e5e87207c1aaafc0990f9d7bc42d8259ceec7f78dcd7464088357c"}
Jul 14 22:11:48.326251 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.326170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"b8d3b1cb1c01b243305abdc78fa3ea785f26d5be3f4c5ddf1c39e51f6c5a7329"}
Jul 14 22:11:48.326251 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.326197    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"64a50a32ea0a8b2b50aa8b743a0332d5e878f76af5216778c7ce8260cbbf3869"}
Jul 14 22:11:48.326251 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.326222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"84de2809-7e09-4f7d-819d-743129c347a7","Type":"ContainerStarted","Data":"adbbe90264650d4496faf2664b221ace4a63cb6d9ca10a35bcb396cc81f9c859"}
Jul 14 22:11:48.351904 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.351851    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-d85979485-bwvm9" podStartSLOduration=1.375247484 podStartE2EDuration="3.35183132s" totalImagesPullingTime="1.976583836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:11:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:11:48.349971676 +0000 UTC m=+149.235817253" watchObservedRunningTime="2026-07-14 22:11:48.35183132 +0000 UTC m=+149.237676899"
Jul 14 22:11:48.379552 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:48.379498    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.361935398 podStartE2EDuration="2.379480817s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.545419ms" isStatefulPod=false podCreationTimestamp="2026-07-14 22:11:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:11:48.378534063 +0000 UTC m=+149.264379652" watchObservedRunningTime="2026-07-14 22:11:48.379480817 +0000 UTC m=+149.265326394"
Jul 14 22:11:51.693649 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:11:51.693601    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:12:46.693511 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:12:46.693472    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:12:46.709051 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:12:46.709023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:12:47.578431 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:12:47.578397    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 22:14:19.475263 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:14:19.475234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:14:19.477058 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:14:19.477038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:14:19.478141 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:14:19.478120    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 22:16:05.419561 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.419526    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jbdpq"]
Jul 14 22:16:05.422697 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.422679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.425666 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.425642    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 14 22:16:05.425666 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.425667    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-h8t7h\""
Jul 14 22:16:05.427018 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.427001    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 14 22:16:05.439492 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.439466    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jbdpq"]
Jul 14 22:16:05.484902 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.484869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfwf7\" (UniqueName: \"kubernetes.io/projected/0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa-kube-api-access-jfwf7\") pod \"cert-manager-55b4d954d5-jbdpq\" (UID: \"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa\") " pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.485075 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.484926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa-bound-sa-token\") pod \"cert-manager-55b4d954d5-jbdpq\" (UID: \"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa\") " pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.585809 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.585768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa-bound-sa-token\") pod \"cert-manager-55b4d954d5-jbdpq\" (UID: \"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa\") " pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.586018 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.585830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwf7\" (UniqueName: \"kubernetes.io/projected/0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa-kube-api-access-jfwf7\") pod \"cert-manager-55b4d954d5-jbdpq\" (UID: \"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa\") " pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.599329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.599300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfwf7\" (UniqueName: \"kubernetes.io/projected/0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa-kube-api-access-jfwf7\") pod \"cert-manager-55b4d954d5-jbdpq\" (UID: \"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa\") " pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.600242 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.600222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa-bound-sa-token\") pod \"cert-manager-55b4d954d5-jbdpq\" (UID: \"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa\") " pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.744013 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.743923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-jbdpq"
Jul 14 22:16:05.892502 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.892293    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-jbdpq"]
Jul 14 22:16:05.895487 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:16:05.895460    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ab483d4_8e7a_4669_ac5b_27f2c11eb8aa.slice/crio-962a6c2071da798d1f5d92f097bf550458da03b4f434c4d868808ce6c8bd6b90 WatchSource:0}: Error finding container 962a6c2071da798d1f5d92f097bf550458da03b4f434c4d868808ce6c8bd6b90: Status 404 returned error can't find the container with id 962a6c2071da798d1f5d92f097bf550458da03b4f434c4d868808ce6c8bd6b90
Jul 14 22:16:05.897266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:05.897249    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:16:06.300427 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:06.300386    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jbdpq" event={"ID":"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa","Type":"ContainerStarted","Data":"962a6c2071da798d1f5d92f097bf550458da03b4f434c4d868808ce6c8bd6b90"}
Jul 14 22:16:10.321558 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:10.321521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-jbdpq" event={"ID":"0ab483d4-8e7a-4669-ac5b-27f2c11eb8aa","Type":"ContainerStarted","Data":"8427f0ec4f0d50ec0d7f21616b6ec70db38437ff4ff6b71f8a165388479d9eec"}
Jul 14 22:16:10.345121 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:10.345057    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-jbdpq" podStartSLOduration=1.9647969170000001 podStartE2EDuration="5.345042289s" totalImagesPullingTime="3.380245372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:16:10.34493768 +0000 UTC m=+411.230783256" watchObservedRunningTime="2026-07-14 22:16:10.345042289 +0000 UTC m=+411.230887866"
Jul 14 22:16:25.944634 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.944605    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"]
Jul 14 22:16:25.950996 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.950976    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:25.954932 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.954912    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 22:16:25.955079 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.955063    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 22:16:25.955446 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.955434    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9kkhx\""
Jul 14 22:16:25.955625 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.955609    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 22:16:25.955675 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.955658    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 22:16:25.984112 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:25.984076    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"]
Jul 14 22:16:26.059584 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.059541    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2bdee43-9359-4fd7-9b12-008715f35cb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.059584 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.059587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2bdee43-9359-4fd7-9b12-008715f35cb1-tmp\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.059837 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.059696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2bdee43-9359-4fd7-9b12-008715f35cb1-manifests\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.059837 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.059744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdr5p\" (UniqueName: \"kubernetes.io/projected/e2bdee43-9359-4fd7-9b12-008715f35cb1-kube-api-access-jdr5p\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.059837 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.059799    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2bdee43-9359-4fd7-9b12-008715f35cb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.160880 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.160840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2bdee43-9359-4fd7-9b12-008715f35cb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.161138 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.160903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2bdee43-9359-4fd7-9b12-008715f35cb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.161138 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.160927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2bdee43-9359-4fd7-9b12-008715f35cb1-tmp\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.161138 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.160966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2bdee43-9359-4fd7-9b12-008715f35cb1-manifests\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.161138 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.160996    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdr5p\" (UniqueName: \"kubernetes.io/projected/e2bdee43-9359-4fd7-9b12-008715f35cb1-kube-api-access-jdr5p\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.161438 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.161406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2bdee43-9359-4fd7-9b12-008715f35cb1-manifests\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.163287 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.163259    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2bdee43-9359-4fd7-9b12-008715f35cb1-tmp\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.163400 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.163366    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2bdee43-9359-4fd7-9b12-008715f35cb1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.163537 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.163516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2bdee43-9359-4fd7-9b12-008715f35cb1-webhook-cert\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.181882 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.181855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdr5p\" (UniqueName: \"kubernetes.io/projected/e2bdee43-9359-4fd7-9b12-008715f35cb1-kube-api-access-jdr5p\") pod \"opendatahub-operator-controller-manager-86c5998494-l2xww\" (UID: \"e2bdee43-9359-4fd7-9b12-008715f35cb1\") " pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.261882 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.261852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:26.405862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:26.405833    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"]
Jul 14 22:16:26.409296 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:16:26.409269    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2bdee43_9359_4fd7_9b12_008715f35cb1.slice/crio-898373f43da814f0a932dc1011812f73ef12771658b2aca1b27f3a32b5959d9f WatchSource:0}: Error finding container 898373f43da814f0a932dc1011812f73ef12771658b2aca1b27f3a32b5959d9f: Status 404 returned error can't find the container with id 898373f43da814f0a932dc1011812f73ef12771658b2aca1b27f3a32b5959d9f
Jul 14 22:16:27.387784 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:27.387746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww" event={"ID":"e2bdee43-9359-4fd7-9b12-008715f35cb1","Type":"ContainerStarted","Data":"898373f43da814f0a932dc1011812f73ef12771658b2aca1b27f3a32b5959d9f"}
Jul 14 22:16:30.402630 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:30.402594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww" event={"ID":"e2bdee43-9359-4fd7-9b12-008715f35cb1","Type":"ContainerStarted","Data":"7e4a20e302924a462f0a3b1211e3177a90354f614d2ea2f637fc011cb2a41e14"}
Jul 14 22:16:31.408456 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:31.408420    2580 generic.go:356] "Generic (PLEG): container finished" podID="e2bdee43-9359-4fd7-9b12-008715f35cb1" containerID="7e4a20e302924a462f0a3b1211e3177a90354f614d2ea2f637fc011cb2a41e14" exitCode=0
Jul 14 22:16:31.408849 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:31.408488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww" event={"ID":"e2bdee43-9359-4fd7-9b12-008715f35cb1","Type":"ContainerDied","Data":"7e4a20e302924a462f0a3b1211e3177a90354f614d2ea2f637fc011cb2a41e14"}
Jul 14 22:16:32.414009 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:32.413973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww" event={"ID":"e2bdee43-9359-4fd7-9b12-008715f35cb1","Type":"ContainerStarted","Data":"140d37102b8c1178449cae4f9abb924488c1591bfff624dff843d09ea825f331"}
Jul 14 22:16:32.414433 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:32.414085    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:32.460933 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:32.460874    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww" podStartSLOduration=2.903448084 podStartE2EDuration="7.460856934s" totalImagesPullingTime="3.463146924s" totalInitContainerRuntime="1.094261926s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:16:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:16:32.459165262 +0000 UTC m=+433.345010838" watchObservedRunningTime="2026-07-14 22:16:32.460856934 +0000 UTC m=+433.346702510"
Jul 14 22:16:43.420163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:43.420078    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86c5998494-l2xww"
Jul 14 22:16:45.072607 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.072573    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"]
Jul 14 22:16:45.075680 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.075664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.080356 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.080335    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mtqf9\""
Jul 14 22:16:45.083410 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.083391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 14 22:16:45.083410 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.083404    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 22:16:45.083563 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.083550    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 14 22:16:45.083605 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.083563    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 22:16:45.083684 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.083666    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 14 22:16:45.089761 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.089741    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"]
Jul 14 22:16:45.122649 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.122622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6597263d-7a30-4b6e-8acc-1bb93ffad911-manager-config\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.122821 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.122655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6597263d-7a30-4b6e-8acc-1bb93ffad911-metrics-cert\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.122821 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.122708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d72h\" (UniqueName: \"kubernetes.io/projected/6597263d-7a30-4b6e-8acc-1bb93ffad911-kube-api-access-4d72h\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.122821 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.122809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6597263d-7a30-4b6e-8acc-1bb93ffad911-cert\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.223959 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.223919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6597263d-7a30-4b6e-8acc-1bb93ffad911-manager-config\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.223959 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.223959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6597263d-7a30-4b6e-8acc-1bb93ffad911-metrics-cert\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.224236 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.224092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d72h\" (UniqueName: \"kubernetes.io/projected/6597263d-7a30-4b6e-8acc-1bb93ffad911-kube-api-access-4d72h\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.224236 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.224232    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6597263d-7a30-4b6e-8acc-1bb93ffad911-cert\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.224716 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.224693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6597263d-7a30-4b6e-8acc-1bb93ffad911-manager-config\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.226498 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.226477    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6597263d-7a30-4b6e-8acc-1bb93ffad911-metrics-cert\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.226571 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.226536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6597263d-7a30-4b6e-8acc-1bb93ffad911-cert\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.247550 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.247520    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d72h\" (UniqueName: \"kubernetes.io/projected/6597263d-7a30-4b6e-8acc-1bb93ffad911-kube-api-access-4d72h\") pod \"lws-controller-manager-66545d98-7dg8n\" (UID: \"6597263d-7a30-4b6e-8acc-1bb93ffad911\") " pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.385800 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.385719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:45.633061 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:45.633040    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"]
Jul 14 22:16:45.634888 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:16:45.634859    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6597263d_7a30_4b6e_8acc_1bb93ffad911.slice/crio-96fe72cec58e749441f182ac13930573d523e27e434f0fd0d7374f2265029fa8 WatchSource:0}: Error finding container 96fe72cec58e749441f182ac13930573d523e27e434f0fd0d7374f2265029fa8: Status 404 returned error can't find the container with id 96fe72cec58e749441f182ac13930573d523e27e434f0fd0d7374f2265029fa8
Jul 14 22:16:46.473887 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.473767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n" event={"ID":"6597263d-7a30-4b6e-8acc-1bb93ffad911","Type":"ContainerStarted","Data":"96fe72cec58e749441f182ac13930573d523e27e434f0fd0d7374f2265029fa8"}
Jul 14 22:16:46.725179 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.725082    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mv56q"]
Jul 14 22:16:46.729047 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.728956    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:46.737399 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.737369    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 14 22:16:46.737399 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.737390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wz2zb\""
Jul 14 22:16:46.800115 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.800024    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mv56q"]
Jul 14 22:16:46.838435 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.838402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:46.838599 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.838450    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57w9c\" (UniqueName: \"kubernetes.io/projected/d07860df-2541-417b-b107-cc2e0aef1ce8-kube-api-access-57w9c\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:46.939274 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.939237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:46.939448 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.939324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57w9c\" (UniqueName: \"kubernetes.io/projected/d07860df-2541-417b-b107-cc2e0aef1ce8-kube-api-access-57w9c\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:46.939448 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:46.939377    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 14 22:16:46.939539 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:46.939462    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert podName:d07860df-2541-417b-b107-cc2e0aef1ce8 nodeName:}" failed. No retries permitted until 2026-07-14 22:16:47.439437972 +0000 UTC m=+448.325283546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert") pod "odh-model-controller-7998c67dc4-mv56q" (UID: "d07860df-2541-417b-b107-cc2e0aef1ce8") : secret "odh-model-controller-webhook-cert" not found
Jul 14 22:16:46.973016 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.972980    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"]
Jul 14 22:16:46.979625 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.979558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:46.986264 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.986212    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 22:16:46.986264 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:46.986239    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 22:16:47.031290 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.031254    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6xwfl\""
Jul 14 22:16:47.032536 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.032495    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 22:16:47.034283 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.034259    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"]
Jul 14 22:16:47.039965 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.039938    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-tmp\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.040118 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.039975    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bvl2\" (UniqueName: \"kubernetes.io/projected/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-kube-api-access-7bvl2\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.040118 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.040012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-tls-certs\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.048087 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.048063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57w9c\" (UniqueName: \"kubernetes.io/projected/d07860df-2541-417b-b107-cc2e0aef1ce8-kube-api-access-57w9c\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:47.070509 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.070481    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 22:16:47.141328 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.141292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-tmp\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.141328 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.141328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bvl2\" (UniqueName: \"kubernetes.io/projected/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-kube-api-access-7bvl2\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.141564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.141360    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-tls-certs\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.143862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.143835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-tmp\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.144071 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.144046    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-tls-certs\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.181279 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.181246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bvl2\" (UniqueName: \"kubernetes.io/projected/bcacde91-8a45-4c37-8d7d-be148ae4e1e1-kube-api-access-7bvl2\") pod \"kube-auth-proxy-5ddb5db5fc-rqdfr\" (UID: \"bcacde91-8a45-4c37-8d7d-be148ae4e1e1\") " pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.290398 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.290358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"
Jul 14 22:16:47.444200 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:47.444161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:47.444377 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:47.444326    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 14 22:16:47.444426 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:47.444391    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert podName:d07860df-2541-417b-b107-cc2e0aef1ce8 nodeName:}" failed. No retries permitted until 2026-07-14 22:16:48.444374897 +0000 UTC m=+449.330220455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert") pod "odh-model-controller-7998c67dc4-mv56q" (UID: "d07860df-2541-417b-b107-cc2e0aef1ce8") : secret "odh-model-controller-webhook-cert" not found
Jul 14 22:16:48.100304 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:16:48.100270    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcacde91_8a45_4c37_8d7d_be148ae4e1e1.slice/crio-a903305c25cb60928076d604d2239c1299a4853e2f8447c374bfefc75de8b33e WatchSource:0}: Error finding container a903305c25cb60928076d604d2239c1299a4853e2f8447c374bfefc75de8b33e: Status 404 returned error can't find the container with id a903305c25cb60928076d604d2239c1299a4853e2f8447c374bfefc75de8b33e
Jul 14 22:16:48.105148 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.105112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr"]
Jul 14 22:16:48.452984 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.452944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:48.455359 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.455342    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d07860df-2541-417b-b107-cc2e0aef1ce8-cert\") pod \"odh-model-controller-7998c67dc4-mv56q\" (UID: \"d07860df-2541-417b-b107-cc2e0aef1ce8\") " pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:48.483671 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.483632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr" event={"ID":"bcacde91-8a45-4c37-8d7d-be148ae4e1e1","Type":"ContainerStarted","Data":"a903305c25cb60928076d604d2239c1299a4853e2f8447c374bfefc75de8b33e"}
Jul 14 22:16:48.485020 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.484996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n" event={"ID":"6597263d-7a30-4b6e-8acc-1bb93ffad911","Type":"ContainerStarted","Data":"5a77e058fdd48d6869c0d5cdc39a29e197b3c0866d9148b9103ae9114cc61a97"}
Jul 14 22:16:48.485183 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.485145    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:16:48.515107 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.515045    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n" podStartSLOduration=1.141744812 podStartE2EDuration="3.515026822s" totalImagesPullingTime="2.37328201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:16:48.514200295 +0000 UTC m=+449.400045871" watchObservedRunningTime="2026-07-14 22:16:48.515026822 +0000 UTC m=+449.400872399"
Jul 14 22:16:48.542918 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.542884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:48.689217 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:48.689190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-mv56q"]
Jul 14 22:16:48.691281 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:16:48.691252    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07860df_2541_417b_b107_cc2e0aef1ce8.slice/crio-1da14c525e2ff519127094314eb788c76ea0cf2a0e12e395fadd8f691f65dab2 WatchSource:0}: Error finding container 1da14c525e2ff519127094314eb788c76ea0cf2a0e12e395fadd8f691f65dab2: Status 404 returned error can't find the container with id 1da14c525e2ff519127094314eb788c76ea0cf2a0e12e395fadd8f691f65dab2
Jul 14 22:16:49.494821 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:49.494747    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q" event={"ID":"d07860df-2541-417b-b107-cc2e0aef1ce8","Type":"ContainerStarted","Data":"1da14c525e2ff519127094314eb788c76ea0cf2a0e12e395fadd8f691f65dab2"}
Jul 14 22:16:52.509572 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:52.509540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr" event={"ID":"bcacde91-8a45-4c37-8d7d-be148ae4e1e1","Type":"ContainerStarted","Data":"cfcd0573daa77a60efb7e9035e7b822277e100e4bf195e38f3226ec42bde4be1"}
Jul 14 22:16:52.511047 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:52.511021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q" event={"ID":"d07860df-2541-417b-b107-cc2e0aef1ce8","Type":"ContainerStarted","Data":"3820c8243750c714cc6359783ca561c9a6e1a4fe63e8b3b2e3cc132534cfbef8"}
Jul 14 22:16:52.511181 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:52.511143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:16:52.539158 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:52.539086    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5ddb5db5fc-rqdfr" podStartSLOduration=2.3345618200000002 podStartE2EDuration="6.539069117s" totalImagesPullingTime="4.204507297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:16:52.537773298 +0000 UTC m=+453.423618875" watchObservedRunningTime="2026-07-14 22:16:52.539069117 +0000 UTC m=+453.424914692"
Jul 14 22:16:52.562681 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:52.562558    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q" podStartSLOduration=2.904599939 podStartE2EDuration="6.562540953s" totalImagesPullingTime="3.657941014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:16:52.561984056 +0000 UTC m=+453.447829634" watchObservedRunningTime="2026-07-14 22:16:52.562540953 +0000 UTC m=+453.448386534"
Jul 14 22:16:59.276325 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.276291    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"]
Jul 14 22:16:59.282449 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.282398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.300819 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:59.300782    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"llmisvc-controller-manager-dockercfg-rxflx\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rxflx\"" type="*v1.Secret"
Jul 14 22:16:59.301277 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:59.301252    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"llmisvc-webhook-server-cert\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" type="*v1.Secret"
Jul 14 22:16:59.301389 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:16:59.301367    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="86156926-0af8-4e53-ae27-de6cc9f11687" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.347166 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.347129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86156926-0af8-4e53-ae27-de6cc9f11687-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.347166 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.347169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cr9j\" (UniqueName: \"kubernetes.io/projected/86156926-0af8-4e53-ae27-de6cc9f11687-kube-api-access-8cr9j\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.397461 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.397420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"]
Jul 14 22:16:59.447937 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.447904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86156926-0af8-4e53-ae27-de6cc9f11687-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.447937 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.447942    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cr9j\" (UniqueName: \"kubernetes.io/projected/86156926-0af8-4e53-ae27-de6cc9f11687-kube-api-access-8cr9j\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.474593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.474564    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cr9j\" (UniqueName: \"kubernetes.io/projected/86156926-0af8-4e53-ae27-de6cc9f11687-kube-api-access-8cr9j\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:16:59.497320 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:16:59.497293    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-66545d98-7dg8n"
Jul 14 22:17:00.448864 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:17:00.448830    2580 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: failed to sync secret cache: timed out waiting for the condition
Jul 14 22:17:00.449284 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:17:00.448910    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86156926-0af8-4e53-ae27-de6cc9f11687-cert podName:86156926-0af8-4e53-ae27-de6cc9f11687 nodeName:}" failed. No retries permitted until 2026-07-14 22:17:00.948892633 +0000 UTC m=+461.834738190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/86156926-0af8-4e53-ae27-de6cc9f11687-cert") pod "llmisvc-controller-manager-777c6bd7d5-p5vwz" (UID: "86156926-0af8-4e53-ae27-de6cc9f11687") : failed to sync secret cache: timed out waiting for the condition
Jul 14 22:17:00.602999 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:00.602969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 14 22:17:00.783579 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:00.783551    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-rxflx\""
Jul 14 22:17:00.961365 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:00.961331    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86156926-0af8-4e53-ae27-de6cc9f11687-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:17:00.963714 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:00.963682    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86156926-0af8-4e53-ae27-de6cc9f11687-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-p5vwz\" (UID: \"86156926-0af8-4e53-ae27-de6cc9f11687\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:17:01.093149 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:01.093013    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:17:01.260003 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:01.259977    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"]
Jul 14 22:17:01.263383 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:17:01.263348    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod86156926_0af8_4e53_ae27_de6cc9f11687.slice/crio-938bcaaa1e1f711a4aaf1bc4f538a179e574cedb0f6793c35468be73b9c3def9 WatchSource:0}: Error finding container 938bcaaa1e1f711a4aaf1bc4f538a179e574cedb0f6793c35468be73b9c3def9: Status 404 returned error can't find the container with id 938bcaaa1e1f711a4aaf1bc4f538a179e574cedb0f6793c35468be73b9c3def9
Jul 14 22:17:01.549679 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:01.549647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz" event={"ID":"86156926-0af8-4e53-ae27-de6cc9f11687","Type":"ContainerStarted","Data":"938bcaaa1e1f711a4aaf1bc4f538a179e574cedb0f6793c35468be73b9c3def9"}
Jul 14 22:17:03.517440 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:03.517409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-mv56q"
Jul 14 22:17:05.571191 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:05.571148    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz" event={"ID":"86156926-0af8-4e53-ae27-de6cc9f11687","Type":"ContainerStarted","Data":"7bc534f0625f32b96a6517aee41c71c908125edc5909f8f683e57a693963a67c"}
Jul 14 22:17:05.571603 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:05.571219    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:17:05.600177 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:05.600093    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz" podStartSLOduration=3.044719836 podStartE2EDuration="6.600076848s" totalImagesPullingTime="3.555357012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:16:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:17:05.599368789 +0000 UTC m=+466.485214365" watchObservedRunningTime="2026-07-14 22:17:05.600076848 +0000 UTC m=+466.485922424"
Jul 14 22:17:36.577140 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:17:36.577081    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-p5vwz"
Jul 14 22:18:43.410523 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.410485    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"]
Jul 14 22:18:43.413739 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.413721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.418298 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.418258    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 22:18:43.418298 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.418278    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 22:18:43.418527 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.418398    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 22:18:43.419391 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.419372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mvnzc\""
Jul 14 22:18:43.419545 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.419515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 22:18:43.430960 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.430931    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"]
Jul 14 22:18:43.457243 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.457197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lzp9\" (UniqueName: \"kubernetes.io/projected/e60c1b76-4a53-4279-9795-15b0aec484ab-kube-api-access-7lzp9\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.457412 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.457268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e60c1b76-4a53-4279-9795-15b0aec484ab-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.457412 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.457319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e60c1b76-4a53-4279-9795-15b0aec484ab-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.557826 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.557791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e60c1b76-4a53-4279-9795-15b0aec484ab-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.558027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.557856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lzp9\" (UniqueName: \"kubernetes.io/projected/e60c1b76-4a53-4279-9795-15b0aec484ab-kube-api-access-7lzp9\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.558027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.557900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e60c1b76-4a53-4279-9795-15b0aec484ab-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.558533 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.558506    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e60c1b76-4a53-4279-9795-15b0aec484ab-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.560373 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.560345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e60c1b76-4a53-4279-9795-15b0aec484ab-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.568036 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.568007    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lzp9\" (UniqueName: \"kubernetes.io/projected/e60c1b76-4a53-4279-9795-15b0aec484ab-kube-api-access-7lzp9\") pod \"kuadrant-console-plugin-66b5774985-8q6cj\" (UID: \"e60c1b76-4a53-4279-9795-15b0aec484ab\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.722888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.722810    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"
Jul 14 22:18:43.856062 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.856039    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj"]
Jul 14 22:18:43.857998 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:18:43.857970    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode60c1b76_4a53_4279_9795_15b0aec484ab.slice/crio-ea850d7b99b54a2621b9961014f098b261670bf703cdd119d1a500035fb8a159 WatchSource:0}: Error finding container ea850d7b99b54a2621b9961014f098b261670bf703cdd119d1a500035fb8a159: Status 404 returned error can't find the container with id ea850d7b99b54a2621b9961014f098b261670bf703cdd119d1a500035fb8a159
Jul 14 22:18:43.990561 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:43.990478    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj" event={"ID":"e60c1b76-4a53-4279-9795-15b0aec484ab","Type":"ContainerStarted","Data":"ea850d7b99b54a2621b9961014f098b261670bf703cdd119d1a500035fb8a159"}
Jul 14 22:18:53.994977 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:53.994941    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"]
Jul 14 22:18:54.015309 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.015275    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"]
Jul 14 22:18:54.015490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.015401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.018782 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.018752    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-n4n2r\""
Jul 14 22:18:54.156169 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.156093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27004afa-0851-4572-914c-d1649e97921a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.156367 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.156252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmb6b\" (UniqueName: \"kubernetes.io/projected/27004afa-0851-4572-914c-d1649e97921a-kube-api-access-mmb6b\") pod \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.257147 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.257090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmb6b\" (UniqueName: \"kubernetes.io/projected/27004afa-0851-4572-914c-d1649e97921a-kube-api-access-mmb6b\") pod \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.257343 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.257233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27004afa-0851-4572-914c-d1649e97921a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.257616 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.257589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27004afa-0851-4572-914c-d1649e97921a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.268132 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.268085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmb6b\" (UniqueName: \"kubernetes.io/projected/27004afa-0851-4572-914c-d1649e97921a-kube-api-access-mmb6b\") pod \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.272899 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.272870    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"]
Jul 14 22:18:54.273273 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.273250    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:18:54.276161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.276076    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"]
Jul 14 22:18:54.303974 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.303939    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"]
Jul 14 22:18:54.308556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.308529    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.329750 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.329678    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"]
Jul 14 22:18:54.459027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.458990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zscvl\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.459236 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.459081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtcbb\" (UniqueName: \"kubernetes.io/projected/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-kube-api-access-jtcbb\") pod \"kuadrant-operator-controller-manager-55467d9f7-zscvl\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.560710 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.560611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zscvl\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.560883 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.560717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtcbb\" (UniqueName: \"kubernetes.io/projected/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-kube-api-access-jtcbb\") pod \"kuadrant-operator-controller-manager-55467d9f7-zscvl\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.561117 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.561075    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-zscvl\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.570517 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.570491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtcbb\" (UniqueName: \"kubernetes.io/projected/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-kube-api-access-jtcbb\") pod \"kuadrant-operator-controller-manager-55467d9f7-zscvl\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:18:54.624211 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:18:54.624171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:19:07.056861 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:07.056837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"]
Jul 14 22:19:07.058823 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:19:07.058798    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab41acd_2bad_44cb_b93b_56e3dd10dfeb.slice/crio-aa02cd9046bbdac10d022f4439da0ed7eb0e1536b20a42bb76417c62b10c0dc6 WatchSource:0}: Error finding container aa02cd9046bbdac10d022f4439da0ed7eb0e1536b20a42bb76417c62b10c0dc6: Status 404 returned error can't find the container with id aa02cd9046bbdac10d022f4439da0ed7eb0e1536b20a42bb76417c62b10c0dc6
Jul 14 22:19:07.111390 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:07.111362    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl" event={"ID":"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb","Type":"ContainerStarted","Data":"aa02cd9046bbdac10d022f4439da0ed7eb0e1536b20a42bb76417c62b10c0dc6"}
Jul 14 22:19:08.122834 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:08.122796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj" event={"ID":"e60c1b76-4a53-4279-9795-15b0aec484ab","Type":"ContainerStarted","Data":"52a05a2ec1457ffb409f46df77a63f264b6f0c0c18d10833c6d7dddddff0c1bc"}
Jul 14 22:19:08.147760 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:08.146763    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8q6cj" podStartSLOduration=1.641240633 podStartE2EDuration="25.146739497s" totalImagesPullingTime="23.505498864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:19:08.145158872 +0000 UTC m=+589.031004451" watchObservedRunningTime="2026-07-14 22:19:08.146739497 +0000 UTC m=+589.032585074"
Jul 14 22:19:09.460606 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:09.460560    2580 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-nqfqd_kuadrant-system_27004afa-0851-4572-914c-d1649e97921a_0(8fe5a332a112992d477a0f5857c6ba1220d8245ec67a4d1d687ef8a0c1329a69): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-nqfqd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 14 22:19:09.461012 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:09.460636    2580 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-nqfqd_kuadrant-system_27004afa-0851-4572-914c-d1649e97921a_0(8fe5a332a112992d477a0f5857c6ba1220d8245ec67a4d1d687ef8a0c1329a69): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-nqfqd to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:10.134714 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.134679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:10.137316 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:10.137286    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="27004afa-0851-4572-914c-d1649e97921a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:10.141005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.140984    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:10.143724 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:10.143698    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="27004afa-0851-4572-914c-d1649e97921a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:10.211745 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.211710    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27004afa-0851-4572-914c-d1649e97921a-extensions-socket-volume\") pod \"27004afa-0851-4572-914c-d1649e97921a\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") "
Jul 14 22:19:10.211924 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.211825    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmb6b\" (UniqueName: \"kubernetes.io/projected/27004afa-0851-4572-914c-d1649e97921a-kube-api-access-mmb6b\") pod \"27004afa-0851-4572-914c-d1649e97921a\" (UID: \"27004afa-0851-4572-914c-d1649e97921a\") "
Jul 14 22:19:10.212039 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.212012    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27004afa-0851-4572-914c-d1649e97921a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "27004afa-0851-4572-914c-d1649e97921a" (UID: "27004afa-0851-4572-914c-d1649e97921a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:19:10.213842 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.213815    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27004afa-0851-4572-914c-d1649e97921a-kube-api-access-mmb6b" (OuterVolumeSpecName: "kube-api-access-mmb6b") pod "27004afa-0851-4572-914c-d1649e97921a" (UID: "27004afa-0851-4572-914c-d1649e97921a"). InnerVolumeSpecName "kube-api-access-mmb6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:19:10.312932 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.312890    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmb6b\" (UniqueName: \"kubernetes.io/projected/27004afa-0851-4572-914c-d1649e97921a-kube-api-access-mmb6b\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:19:10.312932 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:10.312925    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27004afa-0851-4572-914c-d1649e97921a-extensions-socket-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:19:11.138905 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:11.138877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:11.142056 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:11.142013    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="27004afa-0851-4572-914c-d1649e97921a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:11.147511 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:11.147479    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-nqfqd\" is forbidden: User \"system:node:ip-10-0-143-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-117.ec2.internal' and this object" podUID="27004afa-0851-4572-914c-d1649e97921a" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-nqfqd"
Jul 14 22:19:11.601152 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:11.601123    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27004afa-0851-4572-914c-d1649e97921a" path="/var/lib/kubelet/pods/27004afa-0851-4572-914c-d1649e97921a/volumes"
Jul 14 22:19:13.149042 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:13.149009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl" event={"ID":"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb","Type":"ContainerStarted","Data":"3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28"}
Jul 14 22:19:13.149467 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:13.149145    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:19:13.176839 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:13.176787    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl" podStartSLOduration=13.923242183 podStartE2EDuration="19.176770178s" totalImagesPullingTime="5.253527995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:19:13.174796805 +0000 UTC m=+594.060642380" watchObservedRunningTime="2026-07-14 22:19:13.176770178 +0000 UTC m=+594.062615753"
Jul 14 22:19:19.512936 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:19.512909    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:19:19.513403 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:19.513304    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:19:24.155629 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:24.155600    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:19:36.680165 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:36.680126    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"]
Jul 14 22:19:36.680629 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:36.680358    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl" podUID="9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" containerName="manager" containerID="cri-o://3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28" gracePeriod=10
Jul 14 22:19:36.963221 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:36.963125    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:19:37.026012 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.025981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtcbb\" (UniqueName: \"kubernetes.io/projected/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-kube-api-access-jtcbb\") pod \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") "
Jul 14 22:19:37.026227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.026068    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-extensions-socket-volume\") pod \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\" (UID: \"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb\") "
Jul 14 22:19:37.026432 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.026402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" (UID: "9ab41acd-2bad-44cb-b93b-56e3dd10dfeb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:19:37.028240 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.028212    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-kube-api-access-jtcbb" (OuterVolumeSpecName: "kube-api-access-jtcbb") pod "9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" (UID: "9ab41acd-2bad-44cb-b93b-56e3dd10dfeb"). InnerVolumeSpecName "kube-api-access-jtcbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:19:37.126850 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.126807    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-extensions-socket-volume\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:19:37.126850 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.126843    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtcbb\" (UniqueName: \"kubernetes.io/projected/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb-kube-api-access-jtcbb\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:19:37.262963 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.262934    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" containerID="3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28" exitCode=0
Jul 14 22:19:37.263158 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.262971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl" event={"ID":"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb","Type":"ContainerDied","Data":"3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28"}
Jul 14 22:19:37.263158 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.263015    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl" event={"ID":"9ab41acd-2bad-44cb-b93b-56e3dd10dfeb","Type":"ContainerDied","Data":"aa02cd9046bbdac10d022f4439da0ed7eb0e1536b20a42bb76417c62b10c0dc6"}
Jul 14 22:19:37.263158 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.263036    2580 scope.go:117] "RemoveContainer" containerID="3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28"
Jul 14 22:19:37.263158 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.262974    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"
Jul 14 22:19:37.272942 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.272926    2580 scope.go:117] "RemoveContainer" containerID="3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28"
Jul 14 22:19:37.273242 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:37.273225    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28\": container with ID starting with 3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28 not found: ID does not exist" containerID="3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28"
Jul 14 22:19:37.273295 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.273252    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28"} err="failed to get container status \"3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28\": rpc error: code = NotFound desc = could not find container \"3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28\": container with ID starting with 3047b3e0f3583567bbdb3a537ba569df5a4196100ddcec4f585276c2aff14e28 not found: ID does not exist"
Jul 14 22:19:37.293795 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.293759    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"]
Jul 14 22:19:37.298239 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.298215    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-zscvl"]
Jul 14 22:19:37.601583 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:37.601513    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" path="/var/lib/kubelet/pods/9ab41acd-2bad-44cb-b93b-56e3dd10dfeb/volumes"
Jul 14 22:19:44.215656 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.215575    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:19:44.216094 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.215959    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" containerName="manager"
Jul 14 22:19:44.216094 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.215972    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" containerName="manager"
Jul 14 22:19:44.216094 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.216031    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab41acd-2bad-44cb-b93b-56e3dd10dfeb" containerName="manager"
Jul 14 22:19:44.218803 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.218789    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.221476 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.221456    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 22:19:44.231072 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.231048    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:19:44.296219 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.296194    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-config-file\") pod \"limitador-limitador-64c94fd64f-wvxdn\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.296391 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.296230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwghx\" (UniqueName: \"kubernetes.io/projected/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-kube-api-access-xwghx\") pod \"limitador-limitador-64c94fd64f-wvxdn\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.317507 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.317474    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:19:44.396800 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.396764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwghx\" (UniqueName: \"kubernetes.io/projected/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-kube-api-access-xwghx\") pod \"limitador-limitador-64c94fd64f-wvxdn\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.397022 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.396863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-config-file\") pod \"limitador-limitador-64c94fd64f-wvxdn\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.397439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.397420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-config-file\") pod \"limitador-limitador-64c94fd64f-wvxdn\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.412127 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.412070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwghx\" (UniqueName: \"kubernetes.io/projected/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-kube-api-access-xwghx\") pod \"limitador-limitador-64c94fd64f-wvxdn\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.528818 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.528783    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:44.670006 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:44.669971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:19:44.671382 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:19:44.671357    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a4ef2b9_6a24_487c_b893_bd3a37bf122e.slice/crio-bad65cf2370780d270fed70e0d769f3125c4a5b64f98513f01917d0afb17503a WatchSource:0}: Error finding container bad65cf2370780d270fed70e0d769f3125c4a5b64f98513f01917d0afb17503a: Status 404 returned error can't find the container with id bad65cf2370780d270fed70e0d769f3125c4a5b64f98513f01917d0afb17503a
Jul 14 22:19:45.063943 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.063911    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-rltvg"]
Jul 14 22:19:45.096823 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.096782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rltvg"]
Jul 14 22:19:45.097006 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.096915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:45.100029 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.100008    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zjcs2\""
Jul 14 22:19:45.204313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.204273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x7l7\" (UniqueName: \"kubernetes.io/projected/db2f42b7-31d4-47f4-9b12-2aa43c498dd2-kube-api-access-5x7l7\") pod \"authorino-58cd955675-rltvg\" (UID: \"db2f42b7-31d4-47f4-9b12-2aa43c498dd2\") " pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:45.302851 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.302821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn" event={"ID":"8a4ef2b9-6a24-487c-b893-bd3a37bf122e","Type":"ContainerStarted","Data":"bad65cf2370780d270fed70e0d769f3125c4a5b64f98513f01917d0afb17503a"}
Jul 14 22:19:45.305286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.305266    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x7l7\" (UniqueName: \"kubernetes.io/projected/db2f42b7-31d4-47f4-9b12-2aa43c498dd2-kube-api-access-5x7l7\") pod \"authorino-58cd955675-rltvg\" (UID: \"db2f42b7-31d4-47f4-9b12-2aa43c498dd2\") " pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:45.330141 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.330054    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x7l7\" (UniqueName: \"kubernetes.io/projected/db2f42b7-31d4-47f4-9b12-2aa43c498dd2-kube-api-access-5x7l7\") pod \"authorino-58cd955675-rltvg\" (UID: \"db2f42b7-31d4-47f4-9b12-2aa43c498dd2\") " pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:45.407209 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.407175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:45.567654 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:45.567627    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-rltvg"]
Jul 14 22:19:45.569630 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:19:45.569596    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb2f42b7_31d4_47f4_9b12_2aa43c498dd2.slice/crio-f4402dfda643e06e7a0bac77385faad0da9e9106c6c8e85d3cf2936c931d93fb WatchSource:0}: Error finding container f4402dfda643e06e7a0bac77385faad0da9e9106c6c8e85d3cf2936c931d93fb: Status 404 returned error can't find the container with id f4402dfda643e06e7a0bac77385faad0da9e9106c6c8e85d3cf2936c931d93fb
Jul 14 22:19:46.313177 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:46.313081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rltvg" event={"ID":"db2f42b7-31d4-47f4-9b12-2aa43c498dd2","Type":"ContainerStarted","Data":"f4402dfda643e06e7a0bac77385faad0da9e9106c6c8e85d3cf2936c931d93fb"}
Jul 14 22:19:49.818741 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:49.818699    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rltvg"]
Jul 14 22:19:50.338544 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:50.338505    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rltvg" event={"ID":"db2f42b7-31d4-47f4-9b12-2aa43c498dd2","Type":"ContainerStarted","Data":"452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192"}
Jul 14 22:19:50.340048 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:50.340009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn" event={"ID":"8a4ef2b9-6a24-487c-b893-bd3a37bf122e","Type":"ContainerStarted","Data":"0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6"}
Jul 14 22:19:50.340215 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:50.340121    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:19:50.359087 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:50.359034    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-rltvg" podStartSLOduration=1.488227848 podStartE2EDuration="5.359019634s" totalImagesPullingTime="3.870791786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:19:50.357743131 +0000 UTC m=+631.243588707" watchObservedRunningTime="2026-07-14 22:19:50.359019634 +0000 UTC m=+631.244865229"
Jul 14 22:19:50.383466 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:50.383417    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn" podStartSLOduration=1.620313915 podStartE2EDuration="6.383400509s" totalImagesPullingTime="4.763086594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:19:50.381226422 +0000 UTC m=+631.267071989" watchObservedRunningTime="2026-07-14 22:19:50.383400509 +0000 UTC m=+631.269246085"
Jul 14 22:19:51.345060 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:51.345023    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-rltvg" podUID="db2f42b7-31d4-47f4-9b12-2aa43c498dd2" containerName="authorino" containerID="cri-o://452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192" gracePeriod=30
Jul 14 22:19:51.586206 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:51.586183    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:51.685704 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:51.685591    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x7l7\" (UniqueName: \"kubernetes.io/projected/db2f42b7-31d4-47f4-9b12-2aa43c498dd2-kube-api-access-5x7l7\") pod \"db2f42b7-31d4-47f4-9b12-2aa43c498dd2\" (UID: \"db2f42b7-31d4-47f4-9b12-2aa43c498dd2\") "
Jul 14 22:19:51.687624 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:51.687602    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db2f42b7-31d4-47f4-9b12-2aa43c498dd2-kube-api-access-5x7l7" (OuterVolumeSpecName: "kube-api-access-5x7l7") pod "db2f42b7-31d4-47f4-9b12-2aa43c498dd2" (UID: "db2f42b7-31d4-47f4-9b12-2aa43c498dd2"). InnerVolumeSpecName "kube-api-access-5x7l7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:19:51.786784 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:51.786748    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x7l7\" (UniqueName: \"kubernetes.io/projected/db2f42b7-31d4-47f4-9b12-2aa43c498dd2-kube-api-access-5x7l7\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:19:52.350437 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.350406    2580 generic.go:356] "Generic (PLEG): container finished" podID="db2f42b7-31d4-47f4-9b12-2aa43c498dd2" containerID="452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192" exitCode=0
Jul 14 22:19:52.350437 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.350442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rltvg" event={"ID":"db2f42b7-31d4-47f4-9b12-2aa43c498dd2","Type":"ContainerDied","Data":"452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192"}
Jul 14 22:19:52.350944 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.350449    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-rltvg"
Jul 14 22:19:52.350944 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.350467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-rltvg" event={"ID":"db2f42b7-31d4-47f4-9b12-2aa43c498dd2","Type":"ContainerDied","Data":"f4402dfda643e06e7a0bac77385faad0da9e9106c6c8e85d3cf2936c931d93fb"}
Jul 14 22:19:52.350944 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.350483    2580 scope.go:117] "RemoveContainer" containerID="452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192"
Jul 14 22:19:52.359533 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.359516    2580 scope.go:117] "RemoveContainer" containerID="452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192"
Jul 14 22:19:52.359807 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:19:52.359784    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192\": container with ID starting with 452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192 not found: ID does not exist" containerID="452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192"
Jul 14 22:19:52.359901 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.359818    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192"} err="failed to get container status \"452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192\": rpc error: code = NotFound desc = could not find container \"452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192\": container with ID starting with 452ce38700e8daaa4a7105298aa8d0bd8e8d8c86ee323a727d1311c784730192 not found: ID does not exist"
Jul 14 22:19:52.376566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.376542    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-rltvg"]
Jul 14 22:19:52.382883 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:52.382858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-rltvg"]
Jul 14 22:19:53.605960 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:19:53.605918    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db2f42b7-31d4-47f4-9b12-2aa43c498dd2" path="/var/lib/kubelet/pods/db2f42b7-31d4-47f4-9b12-2aa43c498dd2/volumes"
Jul 14 22:20:00.794764 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:00.794725    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:20:00.795295 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:00.794968    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn" podUID="8a4ef2b9-6a24-487c-b893-bd3a37bf122e" containerName="limitador" containerID="cri-o://0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6" gracePeriod=30
Jul 14 22:20:00.795703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:00.795542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:20:01.334359 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.334335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:20:01.365794 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.365708    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-config-file\") pod \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") "
Jul 14 22:20:01.365964 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.365817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwghx\" (UniqueName: \"kubernetes.io/projected/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-kube-api-access-xwghx\") pod \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\" (UID: \"8a4ef2b9-6a24-487c-b893-bd3a37bf122e\") "
Jul 14 22:20:01.366132 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.366085    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-config-file" (OuterVolumeSpecName: "config-file") pod "8a4ef2b9-6a24-487c-b893-bd3a37bf122e" (UID: "8a4ef2b9-6a24-487c-b893-bd3a37bf122e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:20:01.368005 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.367981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-kube-api-access-xwghx" (OuterVolumeSpecName: "kube-api-access-xwghx") pod "8a4ef2b9-6a24-487c-b893-bd3a37bf122e" (UID: "8a4ef2b9-6a24-487c-b893-bd3a37bf122e"). InnerVolumeSpecName "kube-api-access-xwghx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:20:01.393267 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.393238    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a4ef2b9-6a24-487c-b893-bd3a37bf122e" containerID="0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6" exitCode=0
Jul 14 22:20:01.393431 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.393284    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"
Jul 14 22:20:01.393431 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.393285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn" event={"ID":"8a4ef2b9-6a24-487c-b893-bd3a37bf122e","Type":"ContainerDied","Data":"0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6"}
Jul 14 22:20:01.393431 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.393355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wvxdn" event={"ID":"8a4ef2b9-6a24-487c-b893-bd3a37bf122e","Type":"ContainerDied","Data":"bad65cf2370780d270fed70e0d769f3125c4a5b64f98513f01917d0afb17503a"}
Jul 14 22:20:01.393431 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.393378    2580 scope.go:117] "RemoveContainer" containerID="0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6"
Jul 14 22:20:01.401884 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.401863    2580 scope.go:117] "RemoveContainer" containerID="0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6"
Jul 14 22:20:01.402185 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:20:01.402164    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6\": container with ID starting with 0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6 not found: ID does not exist" containerID="0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6"
Jul 14 22:20:01.402233 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.402194    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6"} err="failed to get container status \"0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6\": rpc error: code = NotFound desc = could not find container \"0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6\": container with ID starting with 0a6f36cbf6a956e863fb7ae58878da0051bc794bd842afef190dd0595026fec6 not found: ID does not exist"
Jul 14 22:20:01.417197 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.417173    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:20:01.422093 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.422072    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wvxdn"]
Jul 14 22:20:01.467363 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.467331    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-config-file\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:20:01.467363 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.467359    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwghx\" (UniqueName: \"kubernetes.io/projected/8a4ef2b9-6a24-487c-b893-bd3a37bf122e-kube-api-access-xwghx\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:20:01.600790 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:01.600754    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a4ef2b9-6a24-487c-b893-bd3a37bf122e" path="/var/lib/kubelet/pods/8a4ef2b9-6a24-487c-b893-bd3a37bf122e/volumes"
Jul 14 22:20:04.617159 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617130    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"]
Jul 14 22:20:04.617512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617448    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db2f42b7-31d4-47f4-9b12-2aa43c498dd2" containerName="authorino"
Jul 14 22:20:04.617512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617459    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db2f42b7-31d4-47f4-9b12-2aa43c498dd2" containerName="authorino"
Jul 14 22:20:04.617512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617481    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a4ef2b9-6a24-487c-b893-bd3a37bf122e" containerName="limitador"
Jul 14 22:20:04.617512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617486    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a4ef2b9-6a24-487c-b893-bd3a37bf122e" containerName="limitador"
Jul 14 22:20:04.617641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617539    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a4ef2b9-6a24-487c-b893-bd3a37bf122e" containerName="limitador"
Jul 14 22:20:04.617641 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.617546    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db2f42b7-31d4-47f4-9b12-2aa43c498dd2" containerName="authorino"
Jul 14 22:20:04.620243 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.620226    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.624442 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.624419    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 14 22:20:04.624565 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.624476    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 22:20:04.625811 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.625788    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 22:20:04.625937 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.625816    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zkm67\""
Jul 14 22:20:04.638997 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.638976    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"]
Jul 14 22:20:04.696880 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.696849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nssjm\" (UniqueName: \"kubernetes.io/projected/28e77bbc-1b0f-49f2-9e28-5ae5ce417e52-kube-api-access-nssjm\") pod \"postgres-69fb4594cf-mxkpg\" (UID: \"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.697051 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.696909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28e77bbc-1b0f-49f2-9e28-5ae5ce417e52-data\") pod \"postgres-69fb4594cf-mxkpg\" (UID: \"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.798194 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.798152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28e77bbc-1b0f-49f2-9e28-5ae5ce417e52-data\") pod \"postgres-69fb4594cf-mxkpg\" (UID: \"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.798374 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.798306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nssjm\" (UniqueName: \"kubernetes.io/projected/28e77bbc-1b0f-49f2-9e28-5ae5ce417e52-kube-api-access-nssjm\") pod \"postgres-69fb4594cf-mxkpg\" (UID: \"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.798564 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.798543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/28e77bbc-1b0f-49f2-9e28-5ae5ce417e52-data\") pod \"postgres-69fb4594cf-mxkpg\" (UID: \"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.809355 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.809328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nssjm\" (UniqueName: \"kubernetes.io/projected/28e77bbc-1b0f-49f2-9e28-5ae5ce417e52-kube-api-access-nssjm\") pod \"postgres-69fb4594cf-mxkpg\" (UID: \"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:04.930389 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:04.930305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:05.060174 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:05.060148    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"]
Jul 14 22:20:05.061989 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:20:05.061961    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e77bbc_1b0f_49f2_9e28_5ae5ce417e52.slice/crio-ae9e61e69fa111b576b566554f7ffcaaa357a042a7cbeef4f3e56f33cf621895 WatchSource:0}: Error finding container ae9e61e69fa111b576b566554f7ffcaaa357a042a7cbeef4f3e56f33cf621895: Status 404 returned error can't find the container with id ae9e61e69fa111b576b566554f7ffcaaa357a042a7cbeef4f3e56f33cf621895
Jul 14 22:20:05.412927 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:05.412896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg" event={"ID":"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52","Type":"ContainerStarted","Data":"ae9e61e69fa111b576b566554f7ffcaaa357a042a7cbeef4f3e56f33cf621895"}
Jul 14 22:20:10.440852 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:10.440810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg" event={"ID":"28e77bbc-1b0f-49f2-9e28-5ae5ce417e52","Type":"ContainerStarted","Data":"d6fc91d9693cfded3d46623acc048f6f27a048e541c405c33bdeb57ae7798a48"}
Jul 14 22:20:10.441290 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:10.440910    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:10.460883 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:10.460832    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg" podStartSLOduration=1.6441865789999999 podStartE2EDuration="6.46081692s" totalImagesPullingTime="4.816630341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:20:10.458318074 +0000 UTC m=+651.344163650" watchObservedRunningTime="2026-07-14 22:20:10.46081692 +0000 UTC m=+651.346662495"
Jul 14 22:20:16.475185 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:16.475159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mxkpg"
Jul 14 22:20:17.194113 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.194070    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhm4m"]
Jul 14 22:20:17.202806 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.202774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:17.213710 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.213688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zjcs2\""
Jul 14 22:20:17.244696 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.244663    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhm4m"]
Jul 14 22:20:17.315237 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.315203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h29bn\" (UniqueName: \"kubernetes.io/projected/e1330fef-615c-4603-b3c6-293b3948c416-kube-api-access-h29bn\") pod \"authorino-5d4b67c446-mhm4m\" (UID: \"e1330fef-615c-4603-b3c6-293b3948c416\") " pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:17.415908 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.415865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h29bn\" (UniqueName: \"kubernetes.io/projected/e1330fef-615c-4603-b3c6-293b3948c416-kube-api-access-h29bn\") pod \"authorino-5d4b67c446-mhm4m\" (UID: \"e1330fef-615c-4603-b3c6-293b3948c416\") " pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:17.451882 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.451793    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h29bn\" (UniqueName: \"kubernetes.io/projected/e1330fef-615c-4603-b3c6-293b3948c416-kube-api-access-h29bn\") pod \"authorino-5d4b67c446-mhm4m\" (UID: \"e1330fef-615c-4603-b3c6-293b3948c416\") " pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:17.463077 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.463021    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhm4m"]
Jul 14 22:20:17.463405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.463390    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:17.498996 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.498964    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d87f78c5c-d6m45"]
Jul 14 22:20:17.513682 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.513637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:17.515224 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.515179    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d87f78c5c-d6m45"]
Jul 14 22:20:17.617744 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.617714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dqqp\" (UniqueName: \"kubernetes.io/projected/a8651990-ff98-4b14-bf52-083b3d091d69-kube-api-access-7dqqp\") pod \"authorino-7d87f78c5c-d6m45\" (UID: \"a8651990-ff98-4b14-bf52-083b3d091d69\") " pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:17.630011 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.629988    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhm4m"]
Jul 14 22:20:17.632849 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:20:17.632822    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1330fef_615c_4603_b3c6_293b3948c416.slice/crio-ddb6767c48102be55209817c7321656e6ee7007deded59be461d208959dbaea8 WatchSource:0}: Error finding container ddb6767c48102be55209817c7321656e6ee7007deded59be461d208959dbaea8: Status 404 returned error can't find the container with id ddb6767c48102be55209817c7321656e6ee7007deded59be461d208959dbaea8
Jul 14 22:20:17.718712 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.718616    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqqp\" (UniqueName: \"kubernetes.io/projected/a8651990-ff98-4b14-bf52-083b3d091d69-kube-api-access-7dqqp\") pod \"authorino-7d87f78c5c-d6m45\" (UID: \"a8651990-ff98-4b14-bf52-083b3d091d69\") " pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:17.728057 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.728022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dqqp\" (UniqueName: \"kubernetes.io/projected/a8651990-ff98-4b14-bf52-083b3d091d69-kube-api-access-7dqqp\") pod \"authorino-7d87f78c5c-d6m45\" (UID: \"a8651990-ff98-4b14-bf52-083b3d091d69\") " pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:17.826198 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.826162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:17.850219 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.850197    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d87f78c5c-d6m45"]
Jul 14 22:20:17.987325 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:17.987301    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d87f78c5c-d6m45"]
Jul 14 22:20:17.988886 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:20:17.988862    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8651990_ff98_4b14_bf52_083b3d091d69.slice/crio-05b7ba9090f601b3770d8e59e6e30c16569d74311acb9f8d43c9a24f5189ab28 WatchSource:0}: Error finding container 05b7ba9090f601b3770d8e59e6e30c16569d74311acb9f8d43c9a24f5189ab28: Status 404 returned error can't find the container with id 05b7ba9090f601b3770d8e59e6e30c16569d74311acb9f8d43c9a24f5189ab28
Jul 14 22:20:18.478902 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.478869    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhm4m" event={"ID":"e1330fef-615c-4603-b3c6-293b3948c416","Type":"ContainerStarted","Data":"3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b"}
Jul 14 22:20:18.479139 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.478920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhm4m" event={"ID":"e1330fef-615c-4603-b3c6-293b3948c416","Type":"ContainerStarted","Data":"ddb6767c48102be55209817c7321656e6ee7007deded59be461d208959dbaea8"}
Jul 14 22:20:18.479139 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.478916    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mhm4m" podUID="e1330fef-615c-4603-b3c6-293b3948c416" containerName="authorino" containerID="cri-o://3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b" gracePeriod=30
Jul 14 22:20:18.480339 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.480316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d87f78c5c-d6m45" event={"ID":"a8651990-ff98-4b14-bf52-083b3d091d69","Type":"ContainerStarted","Data":"05b7ba9090f601b3770d8e59e6e30c16569d74311acb9f8d43c9a24f5189ab28"}
Jul 14 22:20:18.496833 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.496781    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mhm4m" podStartSLOduration=0.827587941 podStartE2EDuration="1.496765202s" totalImagesPullingTime="669.177261ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:20:18.495966076 +0000 UTC m=+659.381811652" watchObservedRunningTime="2026-07-14 22:20:18.496765202 +0000 UTC m=+659.382610839"
Jul 14 22:20:18.785001 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.784963    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:18.930030 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.929940    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h29bn\" (UniqueName: \"kubernetes.io/projected/e1330fef-615c-4603-b3c6-293b3948c416-kube-api-access-h29bn\") pod \"e1330fef-615c-4603-b3c6-293b3948c416\" (UID: \"e1330fef-615c-4603-b3c6-293b3948c416\") "
Jul 14 22:20:18.931955 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:18.931931    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1330fef-615c-4603-b3c6-293b3948c416-kube-api-access-h29bn" (OuterVolumeSpecName: "kube-api-access-h29bn") pod "e1330fef-615c-4603-b3c6-293b3948c416" (UID: "e1330fef-615c-4603-b3c6-293b3948c416"). InnerVolumeSpecName "kube-api-access-h29bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:20:19.030686 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.030649    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h29bn\" (UniqueName: \"kubernetes.io/projected/e1330fef-615c-4603-b3c6-293b3948c416-kube-api-access-h29bn\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:20:19.486420 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.486383    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d87f78c5c-d6m45" event={"ID":"a8651990-ff98-4b14-bf52-083b3d091d69","Type":"ContainerStarted","Data":"5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11"}
Jul 14 22:20:19.486611 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.486470    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d87f78c5c-d6m45" podUID="a8651990-ff98-4b14-bf52-083b3d091d69" containerName="authorino" containerID="cri-o://5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11" gracePeriod=30
Jul 14 22:20:19.487817 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.487791    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1330fef-615c-4603-b3c6-293b3948c416" containerID="3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b" exitCode=0
Jul 14 22:20:19.487935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.487830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhm4m" event={"ID":"e1330fef-615c-4603-b3c6-293b3948c416","Type":"ContainerDied","Data":"3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b"}
Jul 14 22:20:19.487935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.487850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mhm4m" event={"ID":"e1330fef-615c-4603-b3c6-293b3948c416","Type":"ContainerDied","Data":"ddb6767c48102be55209817c7321656e6ee7007deded59be461d208959dbaea8"}
Jul 14 22:20:19.487935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.487858    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mhm4m"
Jul 14 22:20:19.487935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.487873    2580 scope.go:117] "RemoveContainer" containerID="3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b"
Jul 14 22:20:19.497800 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.497781    2580 scope.go:117] "RemoveContainer" containerID="3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b"
Jul 14 22:20:19.498088 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:20:19.498057    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b\": container with ID starting with 3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b not found: ID does not exist" containerID="3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b"
Jul 14 22:20:19.498164 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.498108    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b"} err="failed to get container status \"3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b\": rpc error: code = NotFound desc = could not find container \"3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b\": container with ID starting with 3d241e0beee8bf4758f4885a55848dfd3aa9c6ebe477804aecfcef5bd79d414b not found: ID does not exist"
Jul 14 22:20:19.504368 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.504325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d87f78c5c-d6m45" podStartSLOduration=1.922548377 podStartE2EDuration="2.504313143s" totalImagesPullingTime="581.764766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:20:19.503167083 +0000 UTC m=+660.389012656" watchObservedRunningTime="2026-07-14 22:20:19.504313143 +0000 UTC m=+660.390158721"
Jul 14 22:20:19.520905 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.520883    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhm4m"]
Jul 14 22:20:19.525534 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.525512    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mhm4m"]
Jul 14 22:20:19.604713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.604603    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1330fef-615c-4603-b3c6-293b3948c416" path="/var/lib/kubelet/pods/e1330fef-615c-4603-b3c6-293b3948c416/volumes"
Jul 14 22:20:19.798774 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.798748    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:19.937588 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.937555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dqqp\" (UniqueName: \"kubernetes.io/projected/a8651990-ff98-4b14-bf52-083b3d091d69-kube-api-access-7dqqp\") pod \"a8651990-ff98-4b14-bf52-083b3d091d69\" (UID: \"a8651990-ff98-4b14-bf52-083b3d091d69\") "
Jul 14 22:20:19.939605 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:19.939573    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8651990-ff98-4b14-bf52-083b3d091d69-kube-api-access-7dqqp" (OuterVolumeSpecName: "kube-api-access-7dqqp") pod "a8651990-ff98-4b14-bf52-083b3d091d69" (UID: "a8651990-ff98-4b14-bf52-083b3d091d69"). InnerVolumeSpecName "kube-api-access-7dqqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:20:20.038924 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.038838    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dqqp\" (UniqueName: \"kubernetes.io/projected/a8651990-ff98-4b14-bf52-083b3d091d69-kube-api-access-7dqqp\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:20:20.494197 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.494079    2580 generic.go:356] "Generic (PLEG): container finished" podID="a8651990-ff98-4b14-bf52-083b3d091d69" containerID="5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11" exitCode=0
Jul 14 22:20:20.494197 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.494153    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d87f78c5c-d6m45"
Jul 14 22:20:20.494197 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.494166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d87f78c5c-d6m45" event={"ID":"a8651990-ff98-4b14-bf52-083b3d091d69","Type":"ContainerDied","Data":"5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11"}
Jul 14 22:20:20.494515 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.494209    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d87f78c5c-d6m45" event={"ID":"a8651990-ff98-4b14-bf52-083b3d091d69","Type":"ContainerDied","Data":"05b7ba9090f601b3770d8e59e6e30c16569d74311acb9f8d43c9a24f5189ab28"}
Jul 14 22:20:20.494515 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.494230    2580 scope.go:117] "RemoveContainer" containerID="5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11"
Jul 14 22:20:20.502851 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.502835    2580 scope.go:117] "RemoveContainer" containerID="5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11"
Jul 14 22:20:20.503145 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:20:20.503128    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11\": container with ID starting with 5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11 not found: ID does not exist" containerID="5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11"
Jul 14 22:20:20.503226 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.503157    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11"} err="failed to get container status \"5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11\": rpc error: code = NotFound desc = could not find container \"5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11\": container with ID starting with 5518bad28da785abceaaf41f84ccaec1fcf347f225b53a3bef269b036d9a2e11 not found: ID does not exist"
Jul 14 22:20:20.516565 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.516537    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d87f78c5c-d6m45"]
Jul 14 22:20:20.525676 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:20.525654    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d87f78c5c-d6m45"]
Jul 14 22:20:21.601286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:21.601256    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8651990-ff98-4b14-bf52-083b3d091d69" path="/var/lib/kubelet/pods/a8651990-ff98-4b14-bf52-083b3d091d69/volumes"
Jul 14 22:20:25.206340 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206304    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-68c6dbc6bc-jdzpg"]
Jul 14 22:20:25.206698 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206654    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8651990-ff98-4b14-bf52-083b3d091d69" containerName="authorino"
Jul 14 22:20:25.206698 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206665    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8651990-ff98-4b14-bf52-083b3d091d69" containerName="authorino"
Jul 14 22:20:25.206698 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206685    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1330fef-615c-4603-b3c6-293b3948c416" containerName="authorino"
Jul 14 22:20:25.206698 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206691    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1330fef-615c-4603-b3c6-293b3948c416" containerName="authorino"
Jul 14 22:20:25.206839 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206748    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8651990-ff98-4b14-bf52-083b3d091d69" containerName="authorino"
Jul 14 22:20:25.206839 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.206757    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1330fef-615c-4603-b3c6-293b3948c416" containerName="authorino"
Jul 14 22:20:25.215266 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.215242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.221221 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.221191    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 14 22:20:25.222441 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.222215    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 14 22:20:25.222715 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.222694    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rqtkv\""
Jul 14 22:20:25.234044 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.234015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c6dbc6bc-jdzpg"]
Jul 14 22:20:25.388757 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.388726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad412ec7-7703-427e-ab6d-b664668ae750-webhook-cert\") pod \"maas-controller-68c6dbc6bc-jdzpg\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") " pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.388935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.388796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgbp\" (UniqueName: \"kubernetes.io/projected/ad412ec7-7703-427e-ab6d-b664668ae750-kube-api-access-wmgbp\") pod \"maas-controller-68c6dbc6bc-jdzpg\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") " pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.476428 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.476346    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5446c8b597-kwgcl"]
Jul 14 22:20:25.478874 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.478853    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.490265 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.490242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad412ec7-7703-427e-ab6d-b664668ae750-webhook-cert\") pod \"maas-controller-68c6dbc6bc-jdzpg\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") " pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.490372 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.490299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgbp\" (UniqueName: \"kubernetes.io/projected/ad412ec7-7703-427e-ab6d-b664668ae750-kube-api-access-wmgbp\") pod \"maas-controller-68c6dbc6bc-jdzpg\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") " pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.492542 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.492523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad412ec7-7703-427e-ab6d-b664668ae750-webhook-cert\") pod \"maas-controller-68c6dbc6bc-jdzpg\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") " pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.505935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.505908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgbp\" (UniqueName: \"kubernetes.io/projected/ad412ec7-7703-427e-ab6d-b664668ae750-kube-api-access-wmgbp\") pod \"maas-controller-68c6dbc6bc-jdzpg\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") " pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.508227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.508202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5446c8b597-kwgcl"]
Jul 14 22:20:25.528193 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.528165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:25.591810 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.591658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxxd4\" (UniqueName: \"kubernetes.io/projected/ca11bf46-31aa-4e63-8f60-4b99df68f88c-kube-api-access-fxxd4\") pod \"maas-controller-5446c8b597-kwgcl\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") " pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.591810 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.591729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca11bf46-31aa-4e63-8f60-4b99df68f88c-webhook-cert\") pod \"maas-controller-5446c8b597-kwgcl\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") " pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.669264 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.669239    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-68c6dbc6bc-jdzpg"]
Jul 14 22:20:25.671331 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:20:25.671309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad412ec7_7703_427e_ab6d_b664668ae750.slice/crio-1d7dfa3a8e9ea28fa22910180153a1ac461f8892f301e02ecfadcaef62735c84 WatchSource:0}: Error finding container 1d7dfa3a8e9ea28fa22910180153a1ac461f8892f301e02ecfadcaef62735c84: Status 404 returned error can't find the container with id 1d7dfa3a8e9ea28fa22910180153a1ac461f8892f301e02ecfadcaef62735c84
Jul 14 22:20:25.692522 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.692499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxxd4\" (UniqueName: \"kubernetes.io/projected/ca11bf46-31aa-4e63-8f60-4b99df68f88c-kube-api-access-fxxd4\") pod \"maas-controller-5446c8b597-kwgcl\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") " pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.692671 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.692542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca11bf46-31aa-4e63-8f60-4b99df68f88c-webhook-cert\") pod \"maas-controller-5446c8b597-kwgcl\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") " pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.694922 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.694906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca11bf46-31aa-4e63-8f60-4b99df68f88c-webhook-cert\") pod \"maas-controller-5446c8b597-kwgcl\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") " pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.702250 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.702232    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxxd4\" (UniqueName: \"kubernetes.io/projected/ca11bf46-31aa-4e63-8f60-4b99df68f88c-kube-api-access-fxxd4\") pod \"maas-controller-5446c8b597-kwgcl\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") " pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.788794 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.788760    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:25.944159 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:25.944068    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5446c8b597-kwgcl"]
Jul 14 22:20:25.946239 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:20:25.946206    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca11bf46_31aa_4e63_8f60_4b99df68f88c.slice/crio-01beb5651bd110a40c3c81813ffcd7734b04df151c37044df92e42a44fb6a430 WatchSource:0}: Error finding container 01beb5651bd110a40c3c81813ffcd7734b04df151c37044df92e42a44fb6a430: Status 404 returned error can't find the container with id 01beb5651bd110a40c3c81813ffcd7734b04df151c37044df92e42a44fb6a430
Jul 14 22:20:26.529016 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:26.528976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5446c8b597-kwgcl" event={"ID":"ca11bf46-31aa-4e63-8f60-4b99df68f88c","Type":"ContainerStarted","Data":"01beb5651bd110a40c3c81813ffcd7734b04df151c37044df92e42a44fb6a430"}
Jul 14 22:20:26.533073 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:26.532993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg" event={"ID":"ad412ec7-7703-427e-ab6d-b664668ae750","Type":"ContainerStarted","Data":"1d7dfa3a8e9ea28fa22910180153a1ac461f8892f301e02ecfadcaef62735c84"}
Jul 14 22:20:28.544918 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:28.544877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg" event={"ID":"ad412ec7-7703-427e-ab6d-b664668ae750","Type":"ContainerStarted","Data":"561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505"}
Jul 14 22:20:28.545426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:28.544989    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:28.546517 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:28.546494    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5446c8b597-kwgcl" event={"ID":"ca11bf46-31aa-4e63-8f60-4b99df68f88c","Type":"ContainerStarted","Data":"ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5"}
Jul 14 22:20:28.546636 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:28.546554    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:28.635520 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:28.635473    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg" podStartSLOduration=1.344358462 podStartE2EDuration="3.635455871s" totalImagesPullingTime="2.291097409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:20:28.63222842 +0000 UTC m=+669.518073998" watchObservedRunningTime="2026-07-14 22:20:28.635455871 +0000 UTC m=+669.521301489"
Jul 14 22:20:28.702521 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:28.702472    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5446c8b597-kwgcl" podStartSLOduration=1.695259406 podStartE2EDuration="3.702453526s" totalImagesPullingTime="2.00719412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:20:28.695268883 +0000 UTC m=+669.581114460" watchObservedRunningTime="2026-07-14 22:20:28.702453526 +0000 UTC m=+669.588299102"
Jul 14 22:20:39.553722 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.553692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:39.554154 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.553896    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:20:39.629718 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.629685    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c6dbc6bc-jdzpg"]
Jul 14 22:20:39.629960 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.629910    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg" podUID="ad412ec7-7703-427e-ab6d-b664668ae750" containerName="manager" containerID="cri-o://561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505" gracePeriod=30
Jul 14 22:20:39.888207 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.888183    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:39.919472 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.919443    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmgbp\" (UniqueName: \"kubernetes.io/projected/ad412ec7-7703-427e-ab6d-b664668ae750-kube-api-access-wmgbp\") pod \"ad412ec7-7703-427e-ab6d-b664668ae750\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") "
Jul 14 22:20:39.919631 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.919556    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad412ec7-7703-427e-ab6d-b664668ae750-webhook-cert\") pod \"ad412ec7-7703-427e-ab6d-b664668ae750\" (UID: \"ad412ec7-7703-427e-ab6d-b664668ae750\") "
Jul 14 22:20:39.921631 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.921599    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad412ec7-7703-427e-ab6d-b664668ae750-kube-api-access-wmgbp" (OuterVolumeSpecName: "kube-api-access-wmgbp") pod "ad412ec7-7703-427e-ab6d-b664668ae750" (UID: "ad412ec7-7703-427e-ab6d-b664668ae750"). InnerVolumeSpecName "kube-api-access-wmgbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:20:39.921744 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:39.921688    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad412ec7-7703-427e-ab6d-b664668ae750-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ad412ec7-7703-427e-ab6d-b664668ae750" (UID: "ad412ec7-7703-427e-ab6d-b664668ae750"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:20:40.020454 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.020423    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad412ec7-7703-427e-ab6d-b664668ae750-webhook-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:20:40.020454 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.020452    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmgbp\" (UniqueName: \"kubernetes.io/projected/ad412ec7-7703-427e-ab6d-b664668ae750-kube-api-access-wmgbp\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:20:40.604836 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.604795    2580 generic.go:356] "Generic (PLEG): container finished" podID="ad412ec7-7703-427e-ab6d-b664668ae750" containerID="561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505" exitCode=0
Jul 14 22:20:40.605274 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.604845    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg"
Jul 14 22:20:40.605274 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.604860    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg" event={"ID":"ad412ec7-7703-427e-ab6d-b664668ae750","Type":"ContainerDied","Data":"561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505"}
Jul 14 22:20:40.605274 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.604900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-68c6dbc6bc-jdzpg" event={"ID":"ad412ec7-7703-427e-ab6d-b664668ae750","Type":"ContainerDied","Data":"1d7dfa3a8e9ea28fa22910180153a1ac461f8892f301e02ecfadcaef62735c84"}
Jul 14 22:20:40.605274 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.604922    2580 scope.go:117] "RemoveContainer" containerID="561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505"
Jul 14 22:20:40.614022 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.614004    2580 scope.go:117] "RemoveContainer" containerID="561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505"
Jul 14 22:20:40.614361 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:20:40.614291    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505\": container with ID starting with 561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505 not found: ID does not exist" containerID="561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505"
Jul 14 22:20:40.614424 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.614371    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505"} err="failed to get container status \"561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505\": rpc error: code = NotFound desc = could not find container \"561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505\": container with ID starting with 561d4ee073c71a7dda3eaa51658f7ece6ae44431ed4291ec7795f40409fda505 not found: ID does not exist"
Jul 14 22:20:40.638842 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.638818    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-68c6dbc6bc-jdzpg"]
Jul 14 22:20:40.643857 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:40.643829    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-68c6dbc6bc-jdzpg"]
Jul 14 22:20:41.601364 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:20:41.601330    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad412ec7-7703-427e-ab6d-b664668ae750" path="/var/lib/kubelet/pods/ad412ec7-7703-427e-ab6d-b664668ae750/volumes"
Jul 14 22:21:00.697157 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.697119    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"]
Jul 14 22:21:00.702141 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.700279    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad412ec7-7703-427e-ab6d-b664668ae750" containerName="manager"
Jul 14 22:21:00.702141 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.700307    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad412ec7-7703-427e-ab6d-b664668ae750" containerName="manager"
Jul 14 22:21:00.702141 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.700527    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad412ec7-7703-427e-ab6d-b664668ae750" containerName="manager"
Jul 14 22:21:00.705315 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.705287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:00.708461 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.708442    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hkrgl\""
Jul 14 22:21:00.708599 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.708448    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 14 22:21:00.716799 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.716774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"]
Jul 14 22:21:00.786747 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.786711    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"]
Jul 14 22:21:00.789816 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.789801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:00.800028 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.800006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fac104-5517-4ad9-a091-a186aafc2d92-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-fkhks\" (UID: \"f5fac104-5517-4ad9-a091-a186aafc2d92\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:00.800154 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.800047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5skh4\" (UniqueName: \"kubernetes.io/projected/f5fac104-5517-4ad9-a091-a186aafc2d92-kube-api-access-5skh4\") pod \"payload-pre-processing-74584bb9f8-fkhks\" (UID: \"f5fac104-5517-4ad9-a091-a186aafc2d92\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:00.803296 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.803270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"]
Jul 14 22:21:00.901283 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.901247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c98sd\" (UniqueName: \"kubernetes.io/projected/c36be2b9-d826-4905-8d53-77d2ca3c45e3-kube-api-access-c98sd\") pod \"payload-processing-f9b8d9b9b-ncdgq\" (UID: \"c36be2b9-d826-4905-8d53-77d2ca3c45e3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:00.901283 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.901287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fac104-5517-4ad9-a091-a186aafc2d92-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-fkhks\" (UID: \"f5fac104-5517-4ad9-a091-a186aafc2d92\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:00.901500 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.901323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c36be2b9-d826-4905-8d53-77d2ca3c45e3-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ncdgq\" (UID: \"c36be2b9-d826-4905-8d53-77d2ca3c45e3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:00.901500 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.901367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5skh4\" (UniqueName: \"kubernetes.io/projected/f5fac104-5517-4ad9-a091-a186aafc2d92-kube-api-access-5skh4\") pod \"payload-pre-processing-74584bb9f8-fkhks\" (UID: \"f5fac104-5517-4ad9-a091-a186aafc2d92\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:00.901956 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.901935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5fac104-5517-4ad9-a091-a186aafc2d92-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-fkhks\" (UID: \"f5fac104-5517-4ad9-a091-a186aafc2d92\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:00.913012 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:00.912980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5skh4\" (UniqueName: \"kubernetes.io/projected/f5fac104-5517-4ad9-a091-a186aafc2d92-kube-api-access-5skh4\") pod \"payload-pre-processing-74584bb9f8-fkhks\" (UID: \"f5fac104-5517-4ad9-a091-a186aafc2d92\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:01.002015 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.001984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c98sd\" (UniqueName: \"kubernetes.io/projected/c36be2b9-d826-4905-8d53-77d2ca3c45e3-kube-api-access-c98sd\") pod \"payload-processing-f9b8d9b9b-ncdgq\" (UID: \"c36be2b9-d826-4905-8d53-77d2ca3c45e3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:01.002215 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.002035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c36be2b9-d826-4905-8d53-77d2ca3c45e3-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ncdgq\" (UID: \"c36be2b9-d826-4905-8d53-77d2ca3c45e3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:01.002763 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.002744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c36be2b9-d826-4905-8d53-77d2ca3c45e3-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-ncdgq\" (UID: \"c36be2b9-d826-4905-8d53-77d2ca3c45e3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:01.015371 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.015341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:01.017191 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.017170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c98sd\" (UniqueName: \"kubernetes.io/projected/c36be2b9-d826-4905-8d53-77d2ca3c45e3-kube-api-access-c98sd\") pod \"payload-processing-f9b8d9b9b-ncdgq\" (UID: \"c36be2b9-d826-4905-8d53-77d2ca3c45e3\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:01.099433 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.099406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:01.190937 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.190905    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"]
Jul 14 22:21:01.193237 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:21:01.193205    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fac104_5517_4ad9_a091_a186aafc2d92.slice/crio-e7b2a0f8ef792fb97a268dfcaa069722872acf2a22deac876b0dfb9feb4bad84 WatchSource:0}: Error finding container e7b2a0f8ef792fb97a268dfcaa069722872acf2a22deac876b0dfb9feb4bad84: Status 404 returned error can't find the container with id e7b2a0f8ef792fb97a268dfcaa069722872acf2a22deac876b0dfb9feb4bad84
Jul 14 22:21:01.266623 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.266594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"]
Jul 14 22:21:01.268881 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:21:01.268858    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36be2b9_d826_4905_8d53_77d2ca3c45e3.slice/crio-4cb4b08042a1c698b28200793a9511c2b7b081f36e2aeb138ca0995dc5234ab2 WatchSource:0}: Error finding container 4cb4b08042a1c698b28200793a9511c2b7b081f36e2aeb138ca0995dc5234ab2: Status 404 returned error can't find the container with id 4cb4b08042a1c698b28200793a9511c2b7b081f36e2aeb138ca0995dc5234ab2
Jul 14 22:21:01.699556 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.699469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq" event={"ID":"c36be2b9-d826-4905-8d53-77d2ca3c45e3","Type":"ContainerStarted","Data":"4cb4b08042a1c698b28200793a9511c2b7b081f36e2aeb138ca0995dc5234ab2"}
Jul 14 22:21:01.700734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:01.700707    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks" event={"ID":"f5fac104-5517-4ad9-a091-a186aafc2d92","Type":"ContainerStarted","Data":"e7b2a0f8ef792fb97a268dfcaa069722872acf2a22deac876b0dfb9feb4bad84"}
Jul 14 22:21:02.707401 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:02.707367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq" event={"ID":"c36be2b9-d826-4905-8d53-77d2ca3c45e3","Type":"ContainerStarted","Data":"8af3406f38b20feff44be45fd928d4e2adf2e0f35731c8234d2e9fe35afc92a3"}
Jul 14 22:21:02.707830 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:02.707478    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:02.709042 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:02.709013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks" event={"ID":"f5fac104-5517-4ad9-a091-a186aafc2d92","Type":"ContainerStarted","Data":"a36bd821f795ad6c6d8f3a98bed2681aa8c45df91ab03bd1005c6539faf4b025"}
Jul 14 22:21:02.709170 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:02.709071    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:02.736049 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:02.735993    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq" podStartSLOduration=1.45873547 podStartE2EDuration="2.735972873s" totalImagesPullingTime="1.277237403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:21:02.733611529 +0000 UTC m=+703.619457115" watchObservedRunningTime="2026-07-14 22:21:02.735972873 +0000 UTC m=+703.621818462"
Jul 14 22:21:02.777718 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:02.777663    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks" podStartSLOduration=1.428252983 podStartE2EDuration="2.777646137s" totalImagesPullingTime="1.349393154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:21:02.777584864 +0000 UTC m=+703.663430442" watchObservedRunningTime="2026-07-14 22:21:02.777646137 +0000 UTC m=+703.663491756"
Jul 14 22:21:13.715828 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:13.715753    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-fkhks"
Jul 14 22:21:13.715828 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:13.715824    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-ncdgq"
Jul 14 22:21:23.518816 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.518784    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"]
Jul 14 22:21:23.523551 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.523529    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.527144 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.527121    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 14 22:21:23.528579 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.528554    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 22:21:23.528700 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.528609    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 22:21:23.528700 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.528648    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lhvtx\""
Jul 14 22:21:23.541331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.541304    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"]
Jul 14 22:21:23.712777 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.712747    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00a51bae-bace-44ad-9ac1-6721905228bf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.712777 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.712778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.713001 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.712826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.713001 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.712847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjgpw\" (UniqueName: \"kubernetes.io/projected/00a51bae-bace-44ad-9ac1-6721905228bf-kube-api-access-fjgpw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.713001 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.712871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.713001 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.712935    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.813567 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.813567 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjgpw\" (UniqueName: \"kubernetes.io/projected/00a51bae-bace-44ad-9ac1-6721905228bf-kube-api-access-fjgpw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.813567 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.813851 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.813851 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00a51bae-bace-44ad-9ac1-6721905228bf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.813851 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.814011 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.813900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.814068 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.814050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.814159 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.814138    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.815781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.815759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00a51bae-bace-44ad-9ac1-6721905228bf-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.816132 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.816112    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00a51bae-bace-44ad-9ac1-6721905228bf-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:23.837999 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:23.837969    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjgpw\" (UniqueName: \"kubernetes.io/projected/00a51bae-bace-44ad-9ac1-6721905228bf-kube-api-access-fjgpw\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2\" (UID: \"00a51bae-bace-44ad-9ac1-6721905228bf\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:24.134434 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:24.134351    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:24.287808 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:24.287783    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"]
Jul 14 22:21:24.289653 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:21:24.289625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a51bae_bace_44ad_9ac1_6721905228bf.slice/crio-24e917ad93d0fa95148fb1ce7819d4adcdb4619e0ad217ec0743c7b471f74a1e WatchSource:0}: Error finding container 24e917ad93d0fa95148fb1ce7819d4adcdb4619e0ad217ec0743c7b471f74a1e: Status 404 returned error can't find the container with id 24e917ad93d0fa95148fb1ce7819d4adcdb4619e0ad217ec0743c7b471f74a1e
Jul 14 22:21:24.291219 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:24.291203    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:21:24.824578 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:24.824541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2" event={"ID":"00a51bae-bace-44ad-9ac1-6721905228bf","Type":"ContainerStarted","Data":"24e917ad93d0fa95148fb1ce7819d4adcdb4619e0ad217ec0743c7b471f74a1e"}
Jul 14 22:21:25.830829 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:25.830745    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2" event={"ID":"00a51bae-bace-44ad-9ac1-6721905228bf","Type":"ContainerStarted","Data":"a90da2e420f83328758a867f65046327e7258ab3350e263ac28f4d1f474dc919"}
Jul 14 22:21:25.868981 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:25.868925    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2" podStartSLOduration=1.6035385039999999 podStartE2EDuration="2.86890795s" totalImagesPullingTime="1.265369446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:21:25.867213257 +0000 UTC m=+726.753058843" watchObservedRunningTime="2026-07-14 22:21:25.86890795 +0000 UTC m=+726.754753526"
Jul 14 22:21:34.135261 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:34.135218    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:34.135662 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:34.135283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:34.147824 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:34.147791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:34.884808 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:34.884776    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2"
Jul 14 22:21:45.787257 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.787223    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"]
Jul 14 22:21:45.789936 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.789907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:45.792997 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.792975    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 14 22:21:45.805948 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.805920    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"]
Jul 14 22:21:45.920008 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.919967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:45.920206 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.920020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:45.920206 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.920128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:45.920206 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.920153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpldx\" (UniqueName: \"kubernetes.io/projected/5feeb23f-ba95-426b-8fef-f791ea8a2836-kube-api-access-zpldx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:45.920339 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.920275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5feeb23f-ba95-426b-8fef-f791ea8a2836-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:45.920374 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:45.920334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021140 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021330 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021330 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021330 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021330 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpldx\" (UniqueName: \"kubernetes.io/projected/5feeb23f-ba95-426b-8fef-f791ea8a2836-kube-api-access-zpldx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021330 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5feeb23f-ba95-426b-8fef-f791ea8a2836-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021621 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021692 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.021749 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.021707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.023617 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.023597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5feeb23f-ba95-426b-8fef-f791ea8a2836-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.023834 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.023758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5feeb23f-ba95-426b-8fef-f791ea8a2836-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.036437 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.036409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpldx\" (UniqueName: \"kubernetes.io/projected/5feeb23f-ba95-426b-8fef-f791ea8a2836-kube-api-access-zpldx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-zpbr8\" (UID: \"5feeb23f-ba95-426b-8fef-f791ea8a2836\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.100243 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.100152    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:46.236481 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.236455    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"]
Jul 14 22:21:46.238044 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:21:46.238017    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5feeb23f_ba95_426b_8fef_f791ea8a2836.slice/crio-1b64cebd02e0bfdf507ef3f22c07c43c6d3ec220c8fc6c2eb716c21349505324 WatchSource:0}: Error finding container 1b64cebd02e0bfdf507ef3f22c07c43c6d3ec220c8fc6c2eb716c21349505324: Status 404 returned error can't find the container with id 1b64cebd02e0bfdf507ef3f22c07c43c6d3ec220c8fc6c2eb716c21349505324
Jul 14 22:21:46.935019 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.934988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8" event={"ID":"5feeb23f-ba95-426b-8fef-f791ea8a2836","Type":"ContainerStarted","Data":"f43d8270047b490d37dd5ee8951cb24bcba6196f83e6a1c5a97693326bfdad3f"}
Jul 14 22:21:46.935530 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.935046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8" event={"ID":"5feeb23f-ba95-426b-8fef-f791ea8a2836","Type":"ContainerStarted","Data":"1b64cebd02e0bfdf507ef3f22c07c43c6d3ec220c8fc6c2eb716c21349505324"}
Jul 14 22:21:46.965365 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:46.965314    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8" podStartSLOduration=1.75026974 podStartE2EDuration="1.96529804s" totalImagesPullingTime="215.0283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:21:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:21:46.964388788 +0000 UTC m=+747.850234363" watchObservedRunningTime="2026-07-14 22:21:46.96529804 +0000 UTC m=+747.851143615"
Jul 14 22:21:56.101258 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:56.101213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:56.101813 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:56.101329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:56.113834 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:56.113805    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:21:57.003644 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:21:57.003620    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-zpbr8"
Jul 14 22:22:25.288442 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.288408    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"]
Jul 14 22:22:25.291984 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.291963    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.295518 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.295496    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 14 22:22:25.308500 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.308477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"]
Jul 14 22:22:25.389502 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.389471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd2kp\" (UniqueName: \"kubernetes.io/projected/33c8600e-67c7-4b75-ae9e-83fb1de8e121-kube-api-access-kd2kp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.389669 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.389513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.389669 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.389536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.389669 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.389570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.389669 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.389631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c8600e-67c7-4b75-ae9e-83fb1de8e121-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.389827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.389705    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490291 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490291 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490303    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd2kp\" (UniqueName: \"kubernetes.io/projected/33c8600e-67c7-4b75-ae9e-83fb1de8e121-kube-api-access-kd2kp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490555 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490555 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490555 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490402    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490555 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490429    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c8600e-67c7-4b75-ae9e-83fb1de8e121-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490839 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.490936 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.490914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.491032 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.491011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.493172 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.493148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33c8600e-67c7-4b75-ae9e-83fb1de8e121-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.493363 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.493345    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33c8600e-67c7-4b75-ae9e-83fb1de8e121-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.499735 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.499706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd2kp\" (UniqueName: \"kubernetes.io/projected/33c8600e-67c7-4b75-ae9e-83fb1de8e121-kube-api-access-kd2kp\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9\" (UID: \"33c8600e-67c7-4b75-ae9e-83fb1de8e121\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.601389 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.601317    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:25.738629 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:25.738600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"]
Jul 14 22:22:25.740064 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:22:25.740033    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33c8600e_67c7_4b75_ae9e_83fb1de8e121.slice/crio-813a7ee10c1c767588dbaa05cea0b753d6343481fac3c257f6352fa62d813f93 WatchSource:0}: Error finding container 813a7ee10c1c767588dbaa05cea0b753d6343481fac3c257f6352fa62d813f93: Status 404 returned error can't find the container with id 813a7ee10c1c767588dbaa05cea0b753d6343481fac3c257f6352fa62d813f93
Jul 14 22:22:26.127402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:26.127361    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9" event={"ID":"33c8600e-67c7-4b75-ae9e-83fb1de8e121","Type":"ContainerStarted","Data":"02b5ea99eabce47d413480fd2fa304f5808b11c81ed63794914683670b603381"}
Jul 14 22:22:26.127402 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:26.127408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9" event={"ID":"33c8600e-67c7-4b75-ae9e-83fb1de8e121","Type":"ContainerStarted","Data":"813a7ee10c1c767588dbaa05cea0b753d6343481fac3c257f6352fa62d813f93"}
Jul 14 22:22:26.151082 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:26.151025    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9" podStartSLOduration=0.974827058 podStartE2EDuration="1.151007977s" totalImagesPullingTime="176.180919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:22:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:22:26.149292337 +0000 UTC m=+787.035137912" watchObservedRunningTime="2026-07-14 22:22:26.151007977 +0000 UTC m=+787.036853552"
Jul 14 22:22:35.601695 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:35.601664    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:35.602054 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:35.601708    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:35.614329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:35.614306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:22:36.185681 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:22:36.185648    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9"
Jul 14 22:24:19.542817 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:19.542737    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:24:19.544448 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:19.544425    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:24:56.419369 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.419326    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5446c8b597-kwgcl"]
Jul 14 22:24:56.421855 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.419580    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5446c8b597-kwgcl" podUID="ca11bf46-31aa-4e63-8f60-4b99df68f88c" containerName="manager" containerID="cri-o://ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5" gracePeriod=30
Jul 14 22:24:56.676212 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.676145    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:24:56.710389 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.710362    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxxd4\" (UniqueName: \"kubernetes.io/projected/ca11bf46-31aa-4e63-8f60-4b99df68f88c-kube-api-access-fxxd4\") pod \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") "
Jul 14 22:24:56.710579 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.710439    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca11bf46-31aa-4e63-8f60-4b99df68f88c-webhook-cert\") pod \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\" (UID: \"ca11bf46-31aa-4e63-8f60-4b99df68f88c\") "
Jul 14 22:24:56.721019 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.720977    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca11bf46-31aa-4e63-8f60-4b99df68f88c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ca11bf46-31aa-4e63-8f60-4b99df68f88c" (UID: "ca11bf46-31aa-4e63-8f60-4b99df68f88c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:24:56.721019 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.720983    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca11bf46-31aa-4e63-8f60-4b99df68f88c-kube-api-access-fxxd4" (OuterVolumeSpecName: "kube-api-access-fxxd4") pod "ca11bf46-31aa-4e63-8f60-4b99df68f88c" (UID: "ca11bf46-31aa-4e63-8f60-4b99df68f88c"). InnerVolumeSpecName "kube-api-access-fxxd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:24:56.812570 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.812526    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca11bf46-31aa-4e63-8f60-4b99df68f88c-webhook-cert\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:24:56.812570 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.812569    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxxd4\" (UniqueName: \"kubernetes.io/projected/ca11bf46-31aa-4e63-8f60-4b99df68f88c-kube-api-access-fxxd4\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:24:56.835748 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.835712    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca11bf46-31aa-4e63-8f60-4b99df68f88c" containerID="ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5" exitCode=0
Jul 14 22:24:56.835889 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.835779    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5446c8b597-kwgcl"
Jul 14 22:24:56.835889 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.835796    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5446c8b597-kwgcl" event={"ID":"ca11bf46-31aa-4e63-8f60-4b99df68f88c","Type":"ContainerDied","Data":"ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5"}
Jul 14 22:24:56.835889 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.835876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5446c8b597-kwgcl" event={"ID":"ca11bf46-31aa-4e63-8f60-4b99df68f88c","Type":"ContainerDied","Data":"01beb5651bd110a40c3c81813ffcd7734b04df151c37044df92e42a44fb6a430"}
Jul 14 22:24:56.835995 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.835897    2580 scope.go:117] "RemoveContainer" containerID="ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5"
Jul 14 22:24:56.845424 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.845398    2580 scope.go:117] "RemoveContainer" containerID="ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5"
Jul 14 22:24:56.845715 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:24:56.845696    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5\": container with ID starting with ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5 not found: ID does not exist" containerID="ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5"
Jul 14 22:24:56.845792 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.845727    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5"} err="failed to get container status \"ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5\": rpc error: code = NotFound desc = could not find container \"ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5\": container with ID starting with ce8c899cb914bd19c8f9b322e8a24eb394ed7916918753c66a6e53e25ef1d2c5 not found: ID does not exist"
Jul 14 22:24:56.861054 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.861028    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5446c8b597-kwgcl"]
Jul 14 22:24:56.866467 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:56.866444    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5446c8b597-kwgcl"]
Jul 14 22:24:57.601609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.601583    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca11bf46-31aa-4e63-8f60-4b99df68f88c" path="/var/lib/kubelet/pods/ca11bf46-31aa-4e63-8f60-4b99df68f88c/volumes"
Jul 14 22:24:57.683058 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.683030    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5446c8b597-js2xc"]
Jul 14 22:24:57.683420 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.683407    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca11bf46-31aa-4e63-8f60-4b99df68f88c" containerName="manager"
Jul 14 22:24:57.683472 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.683421    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca11bf46-31aa-4e63-8f60-4b99df68f88c" containerName="manager"
Jul 14 22:24:57.683515 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.683495    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca11bf46-31aa-4e63-8f60-4b99df68f88c" containerName="manager"
Jul 14 22:24:57.687695 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.687675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.691891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.691871    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 14 22:24:57.691891 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.691886    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 14 22:24:57.692006 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.691904    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rqtkv\""
Jul 14 22:24:57.698217 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.698195    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5446c8b597-js2xc"]
Jul 14 22:24:57.720734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.720702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv9zl\" (UniqueName: \"kubernetes.io/projected/6a822d58-eb7a-4d37-b8cd-0043d6b5207f-kube-api-access-lv9zl\") pod \"maas-controller-5446c8b597-js2xc\" (UID: \"6a822d58-eb7a-4d37-b8cd-0043d6b5207f\") " pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.720905 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.720766    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a822d58-eb7a-4d37-b8cd-0043d6b5207f-webhook-cert\") pod \"maas-controller-5446c8b597-js2xc\" (UID: \"6a822d58-eb7a-4d37-b8cd-0043d6b5207f\") " pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.822403 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.822367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv9zl\" (UniqueName: \"kubernetes.io/projected/6a822d58-eb7a-4d37-b8cd-0043d6b5207f-kube-api-access-lv9zl\") pod \"maas-controller-5446c8b597-js2xc\" (UID: \"6a822d58-eb7a-4d37-b8cd-0043d6b5207f\") " pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.822571 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.822456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a822d58-eb7a-4d37-b8cd-0043d6b5207f-webhook-cert\") pod \"maas-controller-5446c8b597-js2xc\" (UID: \"6a822d58-eb7a-4d37-b8cd-0043d6b5207f\") " pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.824859 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.824830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a822d58-eb7a-4d37-b8cd-0043d6b5207f-webhook-cert\") pod \"maas-controller-5446c8b597-js2xc\" (UID: \"6a822d58-eb7a-4d37-b8cd-0043d6b5207f\") " pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.832110 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.832078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv9zl\" (UniqueName: \"kubernetes.io/projected/6a822d58-eb7a-4d37-b8cd-0043d6b5207f-kube-api-access-lv9zl\") pod \"maas-controller-5446c8b597-js2xc\" (UID: \"6a822d58-eb7a-4d37-b8cd-0043d6b5207f\") " pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:57.997881 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:57.997782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:58.131887 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:58.131864    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5446c8b597-js2xc"]
Jul 14 22:24:58.133881 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:24:58.133845    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a822d58_eb7a_4d37_b8cd_0043d6b5207f.slice/crio-6d8b6648a9e297a9b774c30baf380e39f1a5c101faefe15158ffd099d0a1e67e WatchSource:0}: Error finding container 6d8b6648a9e297a9b774c30baf380e39f1a5c101faefe15158ffd099d0a1e67e: Status 404 returned error can't find the container with id 6d8b6648a9e297a9b774c30baf380e39f1a5c101faefe15158ffd099d0a1e67e
Jul 14 22:24:58.849529 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:58.849493    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5446c8b597-js2xc" event={"ID":"6a822d58-eb7a-4d37-b8cd-0043d6b5207f","Type":"ContainerStarted","Data":"4b774bc583262e64ab5a3f36c9af5df4affe7c3105301e6a5154324c28b05bc3"}
Jul 14 22:24:58.849927 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:58.849547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5446c8b597-js2xc" event={"ID":"6a822d58-eb7a-4d37-b8cd-0043d6b5207f","Type":"ContainerStarted","Data":"6d8b6648a9e297a9b774c30baf380e39f1a5c101faefe15158ffd099d0a1e67e"}
Jul 14 22:24:58.849927 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:58.849585    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:24:58.872731 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:24:58.872677    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5446c8b597-js2xc" podStartSLOduration=1.5222834509999998 podStartE2EDuration="1.872660361s" totalImagesPullingTime="350.37691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:24:58.87004612 +0000 UTC m=+939.755891708" watchObservedRunningTime="2026-07-14 22:24:58.872660361 +0000 UTC m=+939.758505936"
Jul 14 22:25:09.857470 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:25:09.857439    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5446c8b597-js2xc"
Jul 14 22:29:19.573726 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:29:19.573653    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:29:19.574990 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:29:19.574969    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:34:19.611221 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:34:19.611190    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:34:19.613111 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:34:19.613074    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:39:19.639668 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:39:19.639643    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:39:19.642407 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:39:19.642388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:44:19.668596 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:44:19.668565    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:44:19.672669 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:44:19.672649    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:45:09.787653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.787618    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"]
Jul 14 22:45:09.792124 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.792071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:09.795008 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.794973    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7a211570\"/\"openshift-service-ca.crt\""
Jul 14 22:45:09.796329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.796303    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7a211570\"/\"kube-root-ca.crt\""
Jul 14 22:45:09.796329 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.796304    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7a211570\"/\"default-dockercfg-9sbc8\""
Jul 14 22:45:09.796544 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.796418    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7a211570\"/\"e2e-lifecycle-model-7a211570-kserve-self-signed-certs\""
Jul 14 22:45:09.802455 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.802429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"]
Jul 14 22:45:09.972400 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.972362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05d84334-84ba-48d4-8f4d-27af12f2a994-tls-certs\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:09.972602 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.972409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-tmp-dir\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:09.972602 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.972437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4rdh\" (UniqueName: \"kubernetes.io/projected/05d84334-84ba-48d4-8f4d-27af12f2a994-kube-api-access-x4rdh\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:09.972602 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.972495    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-model-cache\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:09.972602 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.972527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-home\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:09.972602 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:09.972545    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-dshm\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074081 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.073988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05d84334-84ba-48d4-8f4d-27af12f2a994-tls-certs\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074081 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-tmp-dir\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074308 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4rdh\" (UniqueName: \"kubernetes.io/projected/05d84334-84ba-48d4-8f4d-27af12f2a994-kube-api-access-x4rdh\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074308 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-model-cache\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074308 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-home\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074308 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-dshm\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074519 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-tmp-dir\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-model-cache\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.074740 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.074718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-home\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.076465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.076436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-dshm\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.076706 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.076690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05d84334-84ba-48d4-8f4d-27af12f2a994-tls-certs\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.084611 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.084588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4rdh\" (UniqueName: \"kubernetes.io/projected/05d84334-84ba-48d4-8f4d-27af12f2a994-kube-api-access-x4rdh\") pod \"e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") " pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.102593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.102563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:10.243253 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.243219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"]
Jul 14 22:45:10.246510 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:45:10.246472    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05d84334_84ba_48d4_8f4d_27af12f2a994.slice/crio-a4a73cdd571882495666d4a2549921f38069e73ca0030e8b154b025fdfdf28f3 WatchSource:0}: Error finding container a4a73cdd571882495666d4a2549921f38069e73ca0030e8b154b025fdfdf28f3: Status 404 returned error can't find the container with id a4a73cdd571882495666d4a2549921f38069e73ca0030e8b154b025fdfdf28f3
Jul 14 22:45:10.578609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.578573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c" event={"ID":"05d84334-84ba-48d4-8f4d-27af12f2a994","Type":"ContainerStarted","Data":"7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9"}
Jul 14 22:45:10.578791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.578623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c" event={"ID":"05d84334-84ba-48d4-8f4d-27af12f2a994","Type":"ContainerStarted","Data":"a4a73cdd571882495666d4a2549921f38069e73ca0030e8b154b025fdfdf28f3"}
Jul 14 22:45:10.613326 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:10.613276    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c" podStartSLOduration=1.613260143 podStartE2EDuration="1.613260143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:45:10.612605638 +0000 UTC m=+2151.498451213" watchObservedRunningTime="2026-07-14 22:45:10.613260143 +0000 UTC m=+2151.499105718"
Jul 14 22:45:20.103579 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:20.103538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:20.104074 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:20.103597    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:20.116270 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:20.116242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:20.637278 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:20.637246    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:45:57.802749 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:45:57.802716    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"]
Jul 14 22:46:12.823980 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:12.823935    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c" podUID="05d84334-84ba-48d4-8f4d-27af12f2a994" containerName="main" containerID="cri-o://7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9" gracePeriod=45
Jul 14 22:46:13.073133 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.073092    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:46:13.167450 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167353    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-tmp-dir\") pod \"05d84334-84ba-48d4-8f4d-27af12f2a994\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") "
Jul 14 22:46:13.167450 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05d84334-84ba-48d4-8f4d-27af12f2a994-tls-certs\") pod \"05d84334-84ba-48d4-8f4d-27af12f2a994\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") "
Jul 14 22:46:13.167653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-model-cache\") pod \"05d84334-84ba-48d4-8f4d-27af12f2a994\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") "
Jul 14 22:46:13.167653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4rdh\" (UniqueName: \"kubernetes.io/projected/05d84334-84ba-48d4-8f4d-27af12f2a994-kube-api-access-x4rdh\") pod \"05d84334-84ba-48d4-8f4d-27af12f2a994\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") "
Jul 14 22:46:13.167653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167535    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-home\") pod \"05d84334-84ba-48d4-8f4d-27af12f2a994\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") "
Jul 14 22:46:13.167653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-dshm\") pod \"05d84334-84ba-48d4-8f4d-27af12f2a994\" (UID: \"05d84334-84ba-48d4-8f4d-27af12f2a994\") "
Jul 14 22:46:13.167869 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167663    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05d84334-84ba-48d4-8f4d-27af12f2a994" (UID: "05d84334-84ba-48d4-8f4d-27af12f2a994"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:46:13.167869 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167821    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-model-cache" (OuterVolumeSpecName: "model-cache") pod "05d84334-84ba-48d4-8f4d-27af12f2a994" (UID: "05d84334-84ba-48d4-8f4d-27af12f2a994"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:46:13.167869 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167841    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:46:13.167995 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.167887    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-home" (OuterVolumeSpecName: "home") pod "05d84334-84ba-48d4-8f4d-27af12f2a994" (UID: "05d84334-84ba-48d4-8f4d-27af12f2a994"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:46:13.169809 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.169773    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-dshm" (OuterVolumeSpecName: "dshm") pod "05d84334-84ba-48d4-8f4d-27af12f2a994" (UID: "05d84334-84ba-48d4-8f4d-27af12f2a994"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:46:13.169927 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.169813    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05d84334-84ba-48d4-8f4d-27af12f2a994-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05d84334-84ba-48d4-8f4d-27af12f2a994" (UID: "05d84334-84ba-48d4-8f4d-27af12f2a994"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:46:13.169927 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.169835    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05d84334-84ba-48d4-8f4d-27af12f2a994-kube-api-access-x4rdh" (OuterVolumeSpecName: "kube-api-access-x4rdh") pod "05d84334-84ba-48d4-8f4d-27af12f2a994" (UID: "05d84334-84ba-48d4-8f4d-27af12f2a994"). InnerVolumeSpecName "kube-api-access-x4rdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:46:13.268554 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.268517    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05d84334-84ba-48d4-8f4d-27af12f2a994-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:46:13.268554 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.268544    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:46:13.268554 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.268554    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4rdh\" (UniqueName: \"kubernetes.io/projected/05d84334-84ba-48d4-8f4d-27af12f2a994-kube-api-access-x4rdh\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:46:13.268554 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.268563    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:46:13.268827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.268571    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05d84334-84ba-48d4-8f4d-27af12f2a994-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:46:13.875703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.875615    2580 generic.go:356] "Generic (PLEG): container finished" podID="05d84334-84ba-48d4-8f4d-27af12f2a994" containerID="7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9" exitCode=0
Jul 14 22:46:13.875703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.875662    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"
Jul 14 22:46:13.876259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.875705    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c" event={"ID":"05d84334-84ba-48d4-8f4d-27af12f2a994","Type":"ContainerDied","Data":"7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9"}
Jul 14 22:46:13.876259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.875752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c" event={"ID":"05d84334-84ba-48d4-8f4d-27af12f2a994","Type":"ContainerDied","Data":"a4a73cdd571882495666d4a2549921f38069e73ca0030e8b154b025fdfdf28f3"}
Jul 14 22:46:13.876259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.875775    2580 scope.go:117] "RemoveContainer" containerID="7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9"
Jul 14 22:46:13.884550 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.884530    2580 scope.go:117] "RemoveContainer" containerID="7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9"
Jul 14 22:46:13.884834 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:46:13.884813    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9\": container with ID starting with 7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9 not found: ID does not exist" containerID="7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9"
Jul 14 22:46:13.884912 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.884841    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9"} err="failed to get container status \"7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9\": rpc error: code = NotFound desc = could not find container \"7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9\": container with ID starting with 7334c09c3018820ea12a2efb4ccdddf556f595a10ca4af48d2582010cf0bd3c9 not found: ID does not exist"
Jul 14 22:46:13.897865 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.897839    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"]
Jul 14 22:46:13.904270 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:13.904245    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7a211570/e2e-lifecycle-model-7a211570-kserve-6f8c5cfd4c-kxh8c"]
Jul 14 22:46:15.600860 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:46:15.600822    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05d84334-84ba-48d4-8f4d-27af12f2a994" path="/var/lib/kubelet/pods/05d84334-84ba-48d4-8f4d-27af12f2a994/volumes"
Jul 14 22:48:14.094465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.094382    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"]
Jul 14 22:48:14.094901 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.094768    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05d84334-84ba-48d4-8f4d-27af12f2a994" containerName="main"
Jul 14 22:48:14.094901 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.094781    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="05d84334-84ba-48d4-8f4d-27af12f2a994" containerName="main"
Jul 14 22:48:14.094901 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.094839    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="05d84334-84ba-48d4-8f4d-27af12f2a994" containerName="main"
Jul 14 22:48:14.097744 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.097725    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.100854 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.100835    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fcf58c\"/\"kube-root-ca.crt\""
Jul 14 22:48:14.102159 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.102134    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fcf58c\"/\"openshift-service-ca.crt\""
Jul 14 22:48:14.102159 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.102148    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fcf58c\"/\"default-dockercfg-bsdz2\""
Jul 14 22:48:14.102358 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.102149    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fcf58c\"/\"test-model-fcf58c-kserve-self-signed-certs\""
Jul 14 22:48:14.107659 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.107639    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"]
Jul 14 22:48:14.203991 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.203957    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-dshm\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.204199 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.204004    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sd5m\" (UniqueName: \"kubernetes.io/projected/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-kube-api-access-4sd5m\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.204199 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.204090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tls-certs\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.204199 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.204150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tmp-dir\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.204328 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.204227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-model-cache\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.204328 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.204256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-home\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-dshm\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305293 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sd5m\" (UniqueName: \"kubernetes.io/projected/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-kube-api-access-4sd5m\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tls-certs\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tmp-dir\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-model-cache\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305573 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-home\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305823 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-model-cache\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305823 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tmp-dir\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.305938 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.305845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-home\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.307650 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.307627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-dshm\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.307743 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.307718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tls-certs\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.317981 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.317956    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sd5m\" (UniqueName: \"kubernetes.io/projected/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-kube-api-access-4sd5m\") pod \"test-model-fcf58c-kserve-c56d4c4bc-fv85m\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") " pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.408512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.408405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:14.545441 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:14.545305    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"]
Jul 14 22:48:14.547750 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:48:14.547720    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a1c3f1e_c1b9_40f8_b592_b5d81b4726b6.slice/crio-96ce2a764eb1c9c539652396bc44d69c6efe43a1345ddb2e3b1caf00ea8ba438 WatchSource:0}: Error finding container 96ce2a764eb1c9c539652396bc44d69c6efe43a1345ddb2e3b1caf00ea8ba438: Status 404 returned error can't find the container with id 96ce2a764eb1c9c539652396bc44d69c6efe43a1345ddb2e3b1caf00ea8ba438
Jul 14 22:48:15.449466 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:15.449428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m" event={"ID":"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6","Type":"ContainerStarted","Data":"ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03"}
Jul 14 22:48:15.449466 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:15.449471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m" event={"ID":"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6","Type":"ContainerStarted","Data":"96ce2a764eb1c9c539652396bc44d69c6efe43a1345ddb2e3b1caf00ea8ba438"}
Jul 14 22:48:15.475954 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:15.475895    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m" podStartSLOduration=1.475879794 podStartE2EDuration="1.475879794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:48:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:48:15.475000736 +0000 UTC m=+2336.360846313" watchObservedRunningTime="2026-07-14 22:48:15.475879794 +0000 UTC m=+2336.361725430"
Jul 14 22:48:24.409321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:24.409266    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:24.409321 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:24.409329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:24.421810 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:24.421783    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:24.506764 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:24.506738    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:48:40.487018 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.486980    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"]
Jul 14 22:48:40.490732 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.490714    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.493856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.493830    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cb7974\"/\"kube-root-ca.crt\""
Jul 14 22:48:40.493992 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.493867    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-cb7974\"/\"openshift-service-ca.crt\""
Jul 14 22:48:40.494999 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.494981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cb7974\"/\"default-dockercfg-qsq4l\""
Jul 14 22:48:40.495174 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.495153    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-cb7974\"/\"test-model-cb7974-kserve-self-signed-certs\""
Jul 14 22:48:40.500262 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.500238    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"]
Jul 14 22:48:40.554797 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.554763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-model-cache\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.554981 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.554805    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tls-certs\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.554981 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.554924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-dshm\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.554981 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.554952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tmp-dir\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.555086 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.555002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cntm4\" (UniqueName: \"kubernetes.io/projected/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-kube-api-access-cntm4\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.555086 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.555022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-home\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.655773 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.655731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-dshm\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.655773 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.655771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tmp-dir\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.655822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cntm4\" (UniqueName: \"kubernetes.io/projected/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-kube-api-access-cntm4\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.655849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-home\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.655887    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-model-cache\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656034 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.656021    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tls-certs\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.656312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tmp-dir\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656331 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.656321    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-model-cache\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.656427 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.656383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-home\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.658120 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.658066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-dshm\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.658427 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.658409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tls-certs\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.665969 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.665948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cntm4\" (UniqueName: \"kubernetes.io/projected/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-kube-api-access-cntm4\") pod \"test-model-cb7974-kserve-676dcb6689-g277q\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") " pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.801296 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.801263    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:40.935026 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:40.934995    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"]
Jul 14 22:48:40.935722 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:48:40.935695    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded8ccf8c_e783_49c5_8f35_57480fa6d1df.slice/crio-170f55d106fe2c96a924d70f455ae1977b42d837c18c894306fedc87d163515c WatchSource:0}: Error finding container 170f55d106fe2c96a924d70f455ae1977b42d837c18c894306fedc87d163515c: Status 404 returned error can't find the container with id 170f55d106fe2c96a924d70f455ae1977b42d837c18c894306fedc87d163515c
Jul 14 22:48:41.576501 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:41.576461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q" event={"ID":"ed8ccf8c-e783-49c5-8f35-57480fa6d1df","Type":"ContainerStarted","Data":"02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb"}
Jul 14 22:48:41.576914 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:41.576508    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q" event={"ID":"ed8ccf8c-e783-49c5-8f35-57480fa6d1df","Type":"ContainerStarted","Data":"170f55d106fe2c96a924d70f455ae1977b42d837c18c894306fedc87d163515c"}
Jul 14 22:48:41.598465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:41.598421    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q" podStartSLOduration=1.598404132 podStartE2EDuration="1.598404132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:48:41.596325342 +0000 UTC m=+2362.482170915" watchObservedRunningTime="2026-07-14 22:48:41.598404132 +0000 UTC m=+2362.484249707"
Jul 14 22:48:50.802384 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:50.802340    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:50.802829 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:50.802404    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:50.814908 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:50.814878    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:48:51.636651 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:48:51.636621    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:49:12.001709 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:12.001669    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"]
Jul 14 22:49:12.922395 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:12.922363    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"]
Jul 14 22:49:19.699036 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:19.699011    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:49:19.704940 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:19.704920    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:49:27.024319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.024270    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m" podUID="7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" containerName="main" containerID="cri-o://ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03" gracePeriod=45
Jul 14 22:49:27.267409 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.267384    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:49:27.400093 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tls-certs\") pod \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") "
Jul 14 22:49:27.400093 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400050    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4sd5m\" (UniqueName: \"kubernetes.io/projected/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-kube-api-access-4sd5m\") pod \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") "
Jul 14 22:49:27.400093 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-model-cache\") pod \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") "
Jul 14 22:49:27.400403 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400157    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-home\") pod \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") "
Jul 14 22:49:27.400403 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400205    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-dshm\") pod \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") "
Jul 14 22:49:27.400403 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tmp-dir\") pod \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\" (UID: \"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6\") "
Jul 14 22:49:27.400617 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400402    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" (UID: "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:27.400617 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400491    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-home" (OuterVolumeSpecName: "home") pod "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" (UID: "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:27.400617 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400569    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" (UID: "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:27.400785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400715    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:27.400785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400737    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:27.400785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.400750    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:27.402354 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.402322    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-dshm" (OuterVolumeSpecName: "dshm") pod "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" (UID: "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:27.402474 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.402440    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" (UID: "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:49:27.402518 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.402464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-kube-api-access-4sd5m" (OuterVolumeSpecName: "kube-api-access-4sd5m") pod "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" (UID: "7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6"). InnerVolumeSpecName "kube-api-access-4sd5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:49:27.501426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.501386    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:27.501426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.501418    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:27.501426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.501430    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sd5m\" (UniqueName: \"kubernetes.io/projected/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6-kube-api-access-4sd5m\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:27.799241 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.799209    2580 generic.go:356] "Generic (PLEG): container finished" podID="7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" containerID="ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03" exitCode=0
Jul 14 22:49:27.799419 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.799259    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"
Jul 14 22:49:27.799419 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.799289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m" event={"ID":"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6","Type":"ContainerDied","Data":"ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03"}
Jul 14 22:49:27.799419 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.799330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m" event={"ID":"7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6","Type":"ContainerDied","Data":"96ce2a764eb1c9c539652396bc44d69c6efe43a1345ddb2e3b1caf00ea8ba438"}
Jul 14 22:49:27.799419 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.799347    2580 scope.go:117] "RemoveContainer" containerID="ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03"
Jul 14 22:49:27.808303 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.808273    2580 scope.go:117] "RemoveContainer" containerID="ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03"
Jul 14 22:49:27.808581 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:49:27.808561    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03\": container with ID starting with ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03 not found: ID does not exist" containerID="ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03"
Jul 14 22:49:27.808665 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.808587    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03"} err="failed to get container status \"ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03\": rpc error: code = NotFound desc = could not find container \"ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03\": container with ID starting with ccd144a5c5fbbc63593c7ae23718f25771b8dd74e18a207d417fa6d190cdab03 not found: ID does not exist"
Jul 14 22:49:27.821188 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.821160    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"]
Jul 14 22:49:27.827398 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.827376    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-fcf58c/test-model-fcf58c-kserve-c56d4c4bc-fv85m"]
Jul 14 22:49:27.942888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:27.942846    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q" podUID="ed8ccf8c-e783-49c5-8f35-57480fa6d1df" containerName="main" containerID="cri-o://02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb" gracePeriod=45
Jul 14 22:49:28.190769 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.190740    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:49:28.308493 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-model-cache\") pod \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") "
Jul 14 22:49:28.308703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308518    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-home\") pod \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") "
Jul 14 22:49:28.308703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tmp-dir\") pod \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") "
Jul 14 22:49:28.308703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308581    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tls-certs\") pod \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") "
Jul 14 22:49:28.308703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-dshm\") pod \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") "
Jul 14 22:49:28.308703 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cntm4\" (UniqueName: \"kubernetes.io/projected/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-kube-api-access-cntm4\") pod \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\" (UID: \"ed8ccf8c-e783-49c5-8f35-57480fa6d1df\") "
Jul 14 22:49:28.308983 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308758    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed8ccf8c-e783-49c5-8f35-57480fa6d1df" (UID: "ed8ccf8c-e783-49c5-8f35-57480fa6d1df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:28.308983 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed8ccf8c-e783-49c5-8f35-57480fa6d1df" (UID: "ed8ccf8c-e783-49c5-8f35-57480fa6d1df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:28.308983 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.308824    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-home" (OuterVolumeSpecName: "home") pod "ed8ccf8c-e783-49c5-8f35-57480fa6d1df" (UID: "ed8ccf8c-e783-49c5-8f35-57480fa6d1df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:28.309121 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.309012    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:28.309121 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.309030    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:28.309121 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.309038    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:28.310987 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.310957    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-dshm" (OuterVolumeSpecName: "dshm") pod "ed8ccf8c-e783-49c5-8f35-57480fa6d1df" (UID: "ed8ccf8c-e783-49c5-8f35-57480fa6d1df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:49:28.310987 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.310961    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed8ccf8c-e783-49c5-8f35-57480fa6d1df" (UID: "ed8ccf8c-e783-49c5-8f35-57480fa6d1df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:49:28.311152 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.311002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-kube-api-access-cntm4" (OuterVolumeSpecName: "kube-api-access-cntm4") pod "ed8ccf8c-e783-49c5-8f35-57480fa6d1df" (UID: "ed8ccf8c-e783-49c5-8f35-57480fa6d1df"). InnerVolumeSpecName "kube-api-access-cntm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:49:28.410514 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.410470    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:28.410514 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.410504    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:28.410514 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.410514    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cntm4\" (UniqueName: \"kubernetes.io/projected/ed8ccf8c-e783-49c5-8f35-57480fa6d1df-kube-api-access-cntm4\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:49:28.805766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.805732    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed8ccf8c-e783-49c5-8f35-57480fa6d1df" containerID="02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb" exitCode=0
Jul 14 22:49:28.805958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.805771    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"
Jul 14 22:49:28.805958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.805811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q" event={"ID":"ed8ccf8c-e783-49c5-8f35-57480fa6d1df","Type":"ContainerDied","Data":"02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb"}
Jul 14 22:49:28.805958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.805856    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q" event={"ID":"ed8ccf8c-e783-49c5-8f35-57480fa6d1df","Type":"ContainerDied","Data":"170f55d106fe2c96a924d70f455ae1977b42d837c18c894306fedc87d163515c"}
Jul 14 22:49:28.805958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.805880    2580 scope.go:117] "RemoveContainer" containerID="02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb"
Jul 14 22:49:28.814884 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.814868    2580 scope.go:117] "RemoveContainer" containerID="02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb"
Jul 14 22:49:28.815189 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:49:28.815162    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb\": container with ID starting with 02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb not found: ID does not exist" containerID="02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb"
Jul 14 22:49:28.815300 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.815201    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb"} err="failed to get container status \"02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb\": rpc error: code = NotFound desc = could not find container \"02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb\": container with ID starting with 02f49c38fba0d4099218264f341fab3710694c077c8bb8998767b3e3feb748bb not found: ID does not exist"
Jul 14 22:49:28.827869 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.827843    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"]
Jul 14 22:49:28.831929 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:28.831905    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-cb7974/test-model-cb7974-kserve-676dcb6689-g277q"]
Jul 14 22:49:29.602317 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:29.602284    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" path="/var/lib/kubelet/pods/7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6/volumes"
Jul 14 22:49:29.602741 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:49:29.602726    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8ccf8c-e783-49c5-8f35-57480fa6d1df" path="/var/lib/kubelet/pods/ed8ccf8c-e783-49c5-8f35-57480fa6d1df/volumes"
Jul 14 22:52:09.588110 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588061    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"]
Jul 14 22:52:09.588713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588685    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8ccf8c-e783-49c5-8f35-57480fa6d1df" containerName="main"
Jul 14 22:52:09.588803 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588716    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8ccf8c-e783-49c5-8f35-57480fa6d1df" containerName="main"
Jul 14 22:52:09.588803 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588741    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" containerName="main"
Jul 14 22:52:09.588803 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588749    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" containerName="main"
Jul 14 22:52:09.588954 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588915    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1c3f1e-c1b9-40f8-b592-b5d81b4726b6" containerName="main"
Jul 14 22:52:09.588954 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.588932    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed8ccf8c-e783-49c5-8f35-57480fa6d1df" containerName="main"
Jul 14 22:52:09.592053 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.592032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.595588 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.595572    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"default-dockercfg-dlfns\""
Jul 14 22:52:09.595588 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.595581    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"kube-root-ca.crt\""
Jul 14 22:52:09.596905 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.596881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"auth-test-model-7ab9a6-kserve-self-signed-certs\""
Jul 14 22:52:09.596999 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.596894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"openshift-service-ca.crt\""
Jul 14 22:52:09.604754 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.604727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"]
Jul 14 22:52:09.694216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.694174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff43f067-703b-4320-91e7-08157865cecf-tls-certs\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.694405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.694227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-home\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.694405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.694257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-model-cache\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.694405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.694322    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-dshm\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.694405 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.694343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rp9h\" (UniqueName: \"kubernetes.io/projected/ff43f067-703b-4320-91e7-08157865cecf-kube-api-access-2rp9h\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.694562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.694424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-tmp-dir\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.794999 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.794965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-tmp-dir\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795226 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff43f067-703b-4320-91e7-08157865cecf-tls-certs\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795226 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-home\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795226 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-model-cache\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795226 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795118    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-dshm\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795226 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rp9h\" (UniqueName: \"kubernetes.io/projected/ff43f067-703b-4320-91e7-08157865cecf-kube-api-access-2rp9h\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795503 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-home\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795503 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-model-cache\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.795618 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.795569    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-tmp-dir\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.797260 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.797235    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-dshm\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.797598 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.797577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff43f067-703b-4320-91e7-08157865cecf-tls-certs\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.804023 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.803990    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rp9h\" (UniqueName: \"kubernetes.io/projected/ff43f067-703b-4320-91e7-08157865cecf-kube-api-access-2rp9h\") pod \"auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") " pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:09.902395 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:09.902297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:10.039199 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:10.039172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"]
Jul 14 22:52:10.040410 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:52:10.040377    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff43f067_703b_4320_91e7_08157865cecf.slice/crio-546e98e6674c433d7ca87aeb6263f9f14c95347a1dd7b0605368f8b0bef79ca6 WatchSource:0}: Error finding container 546e98e6674c433d7ca87aeb6263f9f14c95347a1dd7b0605368f8b0bef79ca6: Status 404 returned error can't find the container with id 546e98e6674c433d7ca87aeb6263f9f14c95347a1dd7b0605368f8b0bef79ca6
Jul 14 22:52:10.573907 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:10.573868    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m" event={"ID":"ff43f067-703b-4320-91e7-08157865cecf","Type":"ContainerStarted","Data":"fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123"}
Jul 14 22:52:10.574069 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:10.573916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m" event={"ID":"ff43f067-703b-4320-91e7-08157865cecf","Type":"ContainerStarted","Data":"546e98e6674c433d7ca87aeb6263f9f14c95347a1dd7b0605368f8b0bef79ca6"}
Jul 14 22:52:10.594215 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:10.594165    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m" podStartSLOduration=1.594150564 podStartE2EDuration="1.594150564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:52:10.591437763 +0000 UTC m=+2571.477283392" watchObservedRunningTime="2026-07-14 22:52:10.594150564 +0000 UTC m=+2571.479996140"
Jul 14 22:52:19.903319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:19.903266    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:19.903753 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:19.903333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:19.916068 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:19.916040    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:20.637459 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:20.637424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:52:39.268982 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.268949    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"]
Jul 14 22:52:39.272362 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.272340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.275491 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.275463    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"openshift-service-ca.crt\""
Jul 14 22:52:39.276834 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.276790    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"kube-root-ca.crt\""
Jul 14 22:52:39.276834 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.276814    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"default-dockercfg-qrjqn\""
Jul 14 22:52:39.277054 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.276935    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"auth-test-model-546296-kserve-self-signed-certs\""
Jul 14 22:52:39.290170 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.290143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"]
Jul 14 22:52:39.377121 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.377062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-model-cache\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.377286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.377130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzgpn\" (UniqueName: \"kubernetes.io/projected/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-kube-api-access-lzgpn\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.377286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.377175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tmp-dir\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.377286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.377230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tls-certs\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.377286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.377271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-home\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.377286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.377285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-dshm\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478521 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-model-cache\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478521 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzgpn\" (UniqueName: \"kubernetes.io/projected/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-kube-api-access-lzgpn\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tmp-dir\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tls-certs\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-home\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-dshm\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.478973 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478950    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-model-cache\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.479033 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.478994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tmp-dir\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.479033 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.479022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-home\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.480929 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.480908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-dshm\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.481265 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.481245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tls-certs\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.488372 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.488348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzgpn\" (UniqueName: \"kubernetes.io/projected/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-kube-api-access-lzgpn\") pod \"auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") " pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.583314 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.583229    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:39.724343 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:39.724314    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"]
Jul 14 22:52:39.725413 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:52:39.725384    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d5b11f0_40ed_484a_9dfe_7a11d7e1966d.slice/crio-f66cbf3e7d5cf2fdb574df6d2035c0799ab527a0f6add65fd0d5fa84ece64bdb WatchSource:0}: Error finding container f66cbf3e7d5cf2fdb574df6d2035c0799ab527a0f6add65fd0d5fa84ece64bdb: Status 404 returned error can't find the container with id f66cbf3e7d5cf2fdb574df6d2035c0799ab527a0f6add65fd0d5fa84ece64bdb
Jul 14 22:52:40.721043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:40.721004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd" event={"ID":"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d","Type":"ContainerStarted","Data":"7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99"}
Jul 14 22:52:40.721043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:40.721051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd" event={"ID":"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d","Type":"ContainerStarted","Data":"f66cbf3e7d5cf2fdb574df6d2035c0799ab527a0f6add65fd0d5fa84ece64bdb"}
Jul 14 22:52:40.740503 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:40.740452    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd" podStartSLOduration=1.740436178 podStartE2EDuration="1.740436178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:52:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:52:40.739353232 +0000 UTC m=+2601.625198822" watchObservedRunningTime="2026-07-14 22:52:40.740436178 +0000 UTC m=+2601.626281784"
Jul 14 22:52:49.584172 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:49.584045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:49.584172 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:49.584146    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:49.601897 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:49.601867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:52:49.773367 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:52:49.773337    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:54:19.733562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:19.733483    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:54:19.740862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:19.740839    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:54:36.461760 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:36.461726    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"]
Jul 14 22:54:37.400327 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:37.400294    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"]
Jul 14 22:54:51.482713 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.482649    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m" podUID="ff43f067-703b-4320-91e7-08157865cecf" containerName="main" containerID="cri-o://fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123" gracePeriod=45
Jul 14 22:54:51.740411 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.740339    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:54:51.842856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.842824    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-model-cache\") pod \"ff43f067-703b-4320-91e7-08157865cecf\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") "
Jul 14 22:54:51.843055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.842866    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff43f067-703b-4320-91e7-08157865cecf-tls-certs\") pod \"ff43f067-703b-4320-91e7-08157865cecf\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") "
Jul 14 22:54:51.843055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.842882    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-home\") pod \"ff43f067-703b-4320-91e7-08157865cecf\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") "
Jul 14 22:54:51.843055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.842900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rp9h\" (UniqueName: \"kubernetes.io/projected/ff43f067-703b-4320-91e7-08157865cecf-kube-api-access-2rp9h\") pod \"ff43f067-703b-4320-91e7-08157865cecf\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") "
Jul 14 22:54:51.843055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.842924    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-dshm\") pod \"ff43f067-703b-4320-91e7-08157865cecf\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") "
Jul 14 22:54:51.843055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843039    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-tmp-dir\") pod \"ff43f067-703b-4320-91e7-08157865cecf\" (UID: \"ff43f067-703b-4320-91e7-08157865cecf\") "
Jul 14 22:54:51.843349 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff43f067-703b-4320-91e7-08157865cecf" (UID: "ff43f067-703b-4320-91e7-08157865cecf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:51.843349 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843201    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-home" (OuterVolumeSpecName: "home") pod "ff43f067-703b-4320-91e7-08157865cecf" (UID: "ff43f067-703b-4320-91e7-08157865cecf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:51.843349 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843272    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff43f067-703b-4320-91e7-08157865cecf" (UID: "ff43f067-703b-4320-91e7-08157865cecf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:51.843513 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843449    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:51.843513 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843464    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:51.843513 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.843478    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:51.845231 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.845208    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff43f067-703b-4320-91e7-08157865cecf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff43f067-703b-4320-91e7-08157865cecf" (UID: "ff43f067-703b-4320-91e7-08157865cecf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:54:51.845431 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.845412    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff43f067-703b-4320-91e7-08157865cecf-kube-api-access-2rp9h" (OuterVolumeSpecName: "kube-api-access-2rp9h") pod "ff43f067-703b-4320-91e7-08157865cecf" (UID: "ff43f067-703b-4320-91e7-08157865cecf"). InnerVolumeSpecName "kube-api-access-2rp9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:54:51.845497 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.845434    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-dshm" (OuterVolumeSpecName: "dshm") pod "ff43f067-703b-4320-91e7-08157865cecf" (UID: "ff43f067-703b-4320-91e7-08157865cecf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:51.944576 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.944542    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff43f067-703b-4320-91e7-08157865cecf-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:51.944576 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.944579    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rp9h\" (UniqueName: \"kubernetes.io/projected/ff43f067-703b-4320-91e7-08157865cecf-kube-api-access-2rp9h\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:51.944787 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:51.944591    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff43f067-703b-4320-91e7-08157865cecf-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:52.348426 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.348392    2580 generic.go:356] "Generic (PLEG): container finished" podID="ff43f067-703b-4320-91e7-08157865cecf" containerID="fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123" exitCode=0
Jul 14 22:54:52.348612 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.348435    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"
Jul 14 22:54:52.348612 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.348475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m" event={"ID":"ff43f067-703b-4320-91e7-08157865cecf","Type":"ContainerDied","Data":"fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123"}
Jul 14 22:54:52.348612 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.348520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m" event={"ID":"ff43f067-703b-4320-91e7-08157865cecf","Type":"ContainerDied","Data":"546e98e6674c433d7ca87aeb6263f9f14c95347a1dd7b0605368f8b0bef79ca6"}
Jul 14 22:54:52.348612 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.348540    2580 scope.go:117] "RemoveContainer" containerID="fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123"
Jul 14 22:54:52.358055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.358036    2580 scope.go:117] "RemoveContainer" containerID="fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123"
Jul 14 22:54:52.358371 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:54:52.358350    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123\": container with ID starting with fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123 not found: ID does not exist" containerID="fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123"
Jul 14 22:54:52.358457 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.358380    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123"} err="failed to get container status \"fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123\": rpc error: code = NotFound desc = could not find container \"fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123\": container with ID starting with fa08d6f843d2a177fc82b7d5af2a573cd5a98658c609ab42b82a9e59d1512123 not found: ID does not exist"
Jul 14 22:54:52.376371 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.376339    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"]
Jul 14 22:54:52.382747 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.382717    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/auth-test-model-7ab9a6-kserve-7cc7448c95-hkj4m"]
Jul 14 22:54:52.420499 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.420457    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd" podUID="9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" containerName="main" containerID="cri-o://7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99" gracePeriod=45
Jul 14 22:54:52.685855 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.685824    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:54:52.750948 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.750913    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tmp-dir\") pod \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") "
Jul 14 22:54:52.751163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.750960    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-model-cache\") pod \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") "
Jul 14 22:54:52.751163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.750991    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzgpn\" (UniqueName: \"kubernetes.io/projected/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-kube-api-access-lzgpn\") pod \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") "
Jul 14 22:54:52.751163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tls-certs\") pod \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") "
Jul 14 22:54:52.751163 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751041    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-dshm\") pod \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") "
Jul 14 22:54:52.751406 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-home\") pod \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\" (UID: \"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d\") "
Jul 14 22:54:52.751406 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" (UID: "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:52.751406 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" (UID: "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:52.751624 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751542    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-home" (OuterVolumeSpecName: "home") pod "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" (UID: "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:52.751624 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751591    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:52.751624 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.751612    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:52.753491 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.753463    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-dshm" (OuterVolumeSpecName: "dshm") pod "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" (UID: "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:54:52.753491 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.753473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-kube-api-access-lzgpn" (OuterVolumeSpecName: "kube-api-access-lzgpn") pod "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" (UID: "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d"). InnerVolumeSpecName "kube-api-access-lzgpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:54:52.753621 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.753492    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" (UID: "9d5b11f0-40ed-484a-9dfe-7a11d7e1966d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:54:52.852957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.852921    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:52.852957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.852949    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzgpn\" (UniqueName: \"kubernetes.io/projected/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-kube-api-access-lzgpn\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:52.852957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.852961    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:52.852957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:52.852969    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:54:53.355632 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.355596    2580 generic.go:356] "Generic (PLEG): container finished" podID="9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" containerID="7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99" exitCode=0
Jul 14 22:54:53.355818 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.355644    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"
Jul 14 22:54:53.355818 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.355641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd" event={"ID":"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d","Type":"ContainerDied","Data":"7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99"}
Jul 14 22:54:53.355818 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.355766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd" event={"ID":"9d5b11f0-40ed-484a-9dfe-7a11d7e1966d","Type":"ContainerDied","Data":"f66cbf3e7d5cf2fdb574df6d2035c0799ab527a0f6add65fd0d5fa84ece64bdb"}
Jul 14 22:54:53.355818 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.355791    2580 scope.go:117] "RemoveContainer" containerID="7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99"
Jul 14 22:54:53.365257 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.365237    2580 scope.go:117] "RemoveContainer" containerID="7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99"
Jul 14 22:54:53.365534 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:54:53.365516    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99\": container with ID starting with 7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99 not found: ID does not exist" containerID="7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99"
Jul 14 22:54:53.365629 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.365541    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99"} err="failed to get container status \"7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99\": rpc error: code = NotFound desc = could not find container \"7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99\": container with ID starting with 7aab7d21d9abe01964f7e9abe974cc58e384ba4ac6029756ba59b49f687faf99 not found: ID does not exist"
Jul 14 22:54:53.378296 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.378265    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"]
Jul 14 22:54:53.381797 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.381770    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-546296/auth-test-model-546296-kserve-7fc9bb9ff5-bmlqd"]
Jul 14 22:54:53.601212 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.601169    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" path="/var/lib/kubelet/pods/9d5b11f0-40ed-484a-9dfe-7a11d7e1966d/volumes"
Jul 14 22:54:53.601677 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:54:53.601655    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff43f067-703b-4320-91e7-08157865cecf" path="/var/lib/kubelet/pods/ff43f067-703b-4320-91e7-08157865cecf/volumes"
Jul 14 22:55:06.083171 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083137    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"]
Jul 14 22:55:06.083589 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083575    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff43f067-703b-4320-91e7-08157865cecf" containerName="main"
Jul 14 22:55:06.083635 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083590    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff43f067-703b-4320-91e7-08157865cecf" containerName="main"
Jul 14 22:55:06.083635 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083599    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" containerName="main"
Jul 14 22:55:06.083635 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083605    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" containerName="main"
Jul 14 22:55:06.083733 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083674    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff43f067-703b-4320-91e7-08157865cecf" containerName="main"
Jul 14 22:55:06.083733 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.083686    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5b11f0-40ed-484a-9dfe-7a11d7e1966d" containerName="main"
Jul 14 22:55:06.086701 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.086686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.089751 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.089727    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"kube-root-ca.crt\""
Jul 14 22:55:06.091113 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.091074    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"default-dockercfg-qrjqn\""
Jul 14 22:55:06.091113 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.091090    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"openshift-service-ca.crt\""
Jul 14 22:55:06.091288 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.091083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"sub-test-model-546296-kserve-self-signed-certs\""
Jul 14 22:55:06.095935 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.095912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"]
Jul 14 22:55:06.175084 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.175037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-tmp-dir\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.175272 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.175157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-model-cache\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.175272 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.175211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d49c88-b630-4a99-86f6-f957e9b1d610-tls-certs\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.175272 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.175256    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-dshm\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.175422 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.175316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l87xj\" (UniqueName: \"kubernetes.io/projected/a1d49c88-b630-4a99-86f6-f957e9b1d610-kube-api-access-l87xj\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.175422 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.175359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-home\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.276568 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.276536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-tmp-dir\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.276766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.276582    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-model-cache\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.276766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.276607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d49c88-b630-4a99-86f6-f957e9b1d610-tls-certs\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.276766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.276636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-dshm\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.276766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.276655    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l87xj\" (UniqueName: \"kubernetes.io/projected/a1d49c88-b630-4a99-86f6-f957e9b1d610-kube-api-access-l87xj\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.276766 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.276674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-home\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.277048 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.277026    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-tmp-dir\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.277148 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.277041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-model-cache\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.277207 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.277174    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-home\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.278983 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.278960    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-dshm\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.279343 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.279322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d49c88-b630-4a99-86f6-f957e9b1d610-tls-certs\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.287259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.287236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l87xj\" (UniqueName: \"kubernetes.io/projected/a1d49c88-b630-4a99-86f6-f957e9b1d610-kube-api-access-l87xj\") pod \"sub-test-model-546296-kserve-6955587b75-98wl9\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") " pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.397059 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.396966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:06.542789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:06.542764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"]
Jul 14 22:55:06.544606 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:55:06.544579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1d49c88_b630_4a99_86f6_f957e9b1d610.slice/crio-eb383aea1a7db935d13ce56e20bc80154d9fea89af97a1fce038dc9635667882 WatchSource:0}: Error finding container eb383aea1a7db935d13ce56e20bc80154d9fea89af97a1fce038dc9635667882: Status 404 returned error can't find the container with id eb383aea1a7db935d13ce56e20bc80154d9fea89af97a1fce038dc9635667882
Jul 14 22:55:07.422620 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:07.422585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9" event={"ID":"a1d49c88-b630-4a99-86f6-f957e9b1d610","Type":"ContainerStarted","Data":"40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4"}
Jul 14 22:55:07.423313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:07.422631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9" event={"ID":"a1d49c88-b630-4a99-86f6-f957e9b1d610","Type":"ContainerStarted","Data":"eb383aea1a7db935d13ce56e20bc80154d9fea89af97a1fce038dc9635667882"}
Jul 14 22:55:07.440687 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:07.440638    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9" podStartSLOduration=1.440620582 podStartE2EDuration="1.440620582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:55:07.44006568 +0000 UTC m=+2748.325911283" watchObservedRunningTime="2026-07-14 22:55:07.440620582 +0000 UTC m=+2748.326466156"
Jul 14 22:55:16.397789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:16.397727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:16.397789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:16.397801    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:16.410575 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:16.410543    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:16.477952 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:16.477902    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:55:59.695880 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:55:59.695843    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"]
Jul 14 22:56:14.716233 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:14.716183    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9" podUID="a1d49c88-b630-4a99-86f6-f957e9b1d610" containerName="main" containerID="cri-o://40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4" gracePeriod=45
Jul 14 22:56:14.974532 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:14.974471    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:56:15.030512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030477    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l87xj\" (UniqueName: \"kubernetes.io/projected/a1d49c88-b630-4a99-86f6-f957e9b1d610-kube-api-access-l87xj\") pod \"a1d49c88-b630-4a99-86f6-f957e9b1d610\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") "
Jul 14 22:56:15.030512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030517    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-tmp-dir\") pod \"a1d49c88-b630-4a99-86f6-f957e9b1d610\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") "
Jul 14 22:56:15.030781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-model-cache\") pod \"a1d49c88-b630-4a99-86f6-f957e9b1d610\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") "
Jul 14 22:56:15.030781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030562    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d49c88-b630-4a99-86f6-f957e9b1d610-tls-certs\") pod \"a1d49c88-b630-4a99-86f6-f957e9b1d610\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") "
Jul 14 22:56:15.030781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-dshm\") pod \"a1d49c88-b630-4a99-86f6-f957e9b1d610\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") "
Jul 14 22:56:15.030781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030668    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-home\") pod \"a1d49c88-b630-4a99-86f6-f957e9b1d610\" (UID: \"a1d49c88-b630-4a99-86f6-f957e9b1d610\") "
Jul 14 22:56:15.030978 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1d49c88-b630-4a99-86f6-f957e9b1d610" (UID: "a1d49c88-b630-4a99-86f6-f957e9b1d610"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:56:15.030978 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1d49c88-b630-4a99-86f6-f957e9b1d610" (UID: "a1d49c88-b630-4a99-86f6-f957e9b1d610"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:56:15.031083 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.030985    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-home" (OuterVolumeSpecName: "home") pod "a1d49c88-b630-4a99-86f6-f957e9b1d610" (UID: "a1d49c88-b630-4a99-86f6-f957e9b1d610"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:56:15.031083 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.031062    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:56:15.031201 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.031083    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:56:15.032854 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.032822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d49c88-b630-4a99-86f6-f957e9b1d610-kube-api-access-l87xj" (OuterVolumeSpecName: "kube-api-access-l87xj") pod "a1d49c88-b630-4a99-86f6-f957e9b1d610" (UID: "a1d49c88-b630-4a99-86f6-f957e9b1d610"). InnerVolumeSpecName "kube-api-access-l87xj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:56:15.032854 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.032848    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1d49c88-b630-4a99-86f6-f957e9b1d610-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1d49c88-b630-4a99-86f6-f957e9b1d610" (UID: "a1d49c88-b630-4a99-86f6-f957e9b1d610"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:56:15.033000 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.032858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-dshm" (OuterVolumeSpecName: "dshm") pod "a1d49c88-b630-4a99-86f6-f957e9b1d610" (UID: "a1d49c88-b630-4a99-86f6-f957e9b1d610"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:56:15.132541 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.132502    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:56:15.132541 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.132533    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1d49c88-b630-4a99-86f6-f957e9b1d610-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:56:15.132541 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.132543    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l87xj\" (UniqueName: \"kubernetes.io/projected/a1d49c88-b630-4a99-86f6-f957e9b1d610-kube-api-access-l87xj\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:56:15.132791 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.132554    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1d49c88-b630-4a99-86f6-f957e9b1d610-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:56:15.748594 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.748560    2580 generic.go:356] "Generic (PLEG): container finished" podID="a1d49c88-b630-4a99-86f6-f957e9b1d610" containerID="40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4" exitCode=0
Jul 14 22:56:15.749076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.748602    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"
Jul 14 22:56:15.749076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.748620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9" event={"ID":"a1d49c88-b630-4a99-86f6-f957e9b1d610","Type":"ContainerDied","Data":"40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4"}
Jul 14 22:56:15.749076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.748651    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9" event={"ID":"a1d49c88-b630-4a99-86f6-f957e9b1d610","Type":"ContainerDied","Data":"eb383aea1a7db935d13ce56e20bc80154d9fea89af97a1fce038dc9635667882"}
Jul 14 22:56:15.749076 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.748666    2580 scope.go:117] "RemoveContainer" containerID="40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4"
Jul 14 22:56:15.757490 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.757467    2580 scope.go:117] "RemoveContainer" containerID="40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4"
Jul 14 22:56:15.757745 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:56:15.757729    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4\": container with ID starting with 40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4 not found: ID does not exist" containerID="40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4"
Jul 14 22:56:15.757789 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.757753    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4"} err="failed to get container status \"40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4\": rpc error: code = NotFound desc = could not find container \"40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4\": container with ID starting with 40fa65d2236d976553f59163191963a6ce479c8a72366a5aec46786712022cd4 not found: ID does not exist"
Jul 14 22:56:15.767113 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.767080    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"]
Jul 14 22:56:15.770191 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:15.770171    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-546296/sub-test-model-546296-kserve-6955587b75-98wl9"]
Jul 14 22:56:17.602208 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:17.602168    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1d49c88-b630-4a99-86f6-f957e9b1d610" path="/var/lib/kubelet/pods/a1d49c88-b630-4a99-86f6-f957e9b1d610/volumes"
Jul 14 22:56:19.691138 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.691083    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"]
Jul 14 22:56:19.691548 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.691511    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1d49c88-b630-4a99-86f6-f957e9b1d610" containerName="main"
Jul 14 22:56:19.691548 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.691524    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1d49c88-b630-4a99-86f6-f957e9b1d610" containerName="main"
Jul 14 22:56:19.691625 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.691591    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1d49c88-b630-4a99-86f6-f957e9b1d610" containerName="main"
Jul 14 22:56:19.696263 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.696246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.699464 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.699434    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"kube-root-ca.crt\""
Jul 14 22:56:19.700820 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.700800    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"default-dockercfg-dlfns\""
Jul 14 22:56:19.700945 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.700831    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"rate-test-model-7ab9a6-kserve-self-signed-certs\""
Jul 14 22:56:19.700945 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.700890    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7ab9a6\"/\"openshift-service-ca.crt\""
Jul 14 22:56:19.704986 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.704966    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"]
Jul 14 22:56:19.777592 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.777561    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-dshm\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.777797 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.777618    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-model-cache\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.777797 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.777666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-tmp-dir\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.777797 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.777697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37069d38-1262-4e3c-92c5-c7da1081ae61-tls-certs\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.777797 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.777728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhl9s\" (UniqueName: \"kubernetes.io/projected/37069d38-1262-4e3c-92c5-c7da1081ae61-kube-api-access-qhl9s\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.777958 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.777830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-home\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.878593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.878536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-model-cache\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.878593 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.878592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-tmp-dir\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.878841 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.878618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37069d38-1262-4e3c-92c5-c7da1081ae61-tls-certs\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.878841 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.878647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhl9s\" (UniqueName: \"kubernetes.io/projected/37069d38-1262-4e3c-92c5-c7da1081ae61-kube-api-access-qhl9s\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.878841 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.878739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-home\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.878841 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.878808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-dshm\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.879118 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.879071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-tmp-dir\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.879208 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.879150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-home\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.879366 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.879346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-model-cache\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.881024 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.881003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-dshm\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.881391 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.881373    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37069d38-1262-4e3c-92c5-c7da1081ae61-tls-certs\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:19.888061 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:19.888036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhl9s\" (UniqueName: \"kubernetes.io/projected/37069d38-1262-4e3c-92c5-c7da1081ae61-kube-api-access-qhl9s\") pod \"rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") " pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:20.006786 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:20.006754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:20.145169 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:20.145145    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"]
Jul 14 22:56:20.146797 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:56:20.146774    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37069d38_1262_4e3c_92c5_c7da1081ae61.slice/crio-41b1925d3fdefc4639fd7090afb1721b63e3159ae0e7ed4e920d49fa527707a7 WatchSource:0}: Error finding container 41b1925d3fdefc4639fd7090afb1721b63e3159ae0e7ed4e920d49fa527707a7: Status 404 returned error can't find the container with id 41b1925d3fdefc4639fd7090afb1721b63e3159ae0e7ed4e920d49fa527707a7
Jul 14 22:56:20.774734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:20.774696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw" event={"ID":"37069d38-1262-4e3c-92c5-c7da1081ae61","Type":"ContainerStarted","Data":"351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447"}
Jul 14 22:56:20.774734 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:20.774739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw" event={"ID":"37069d38-1262-4e3c-92c5-c7da1081ae61","Type":"ContainerStarted","Data":"41b1925d3fdefc4639fd7090afb1721b63e3159ae0e7ed4e920d49fa527707a7"}
Jul 14 22:56:20.797716 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:20.797669    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw" podStartSLOduration=1.797654279 podStartE2EDuration="1.797654279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:56:20.796087014 +0000 UTC m=+2821.681932592" watchObservedRunningTime="2026-07-14 22:56:20.797654279 +0000 UTC m=+2821.683499855"
Jul 14 22:56:30.007366 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:30.007328    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:30.007765 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:30.007387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:30.019990 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:30.019964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:30.834974 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:30.834937    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:56:39.680930 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.680896    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"]
Jul 14 22:56:39.684484 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.684461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.688894 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.688872    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"default-dockercfg-qrjqn\""
Jul 14 22:56:39.689027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.688944    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"openshift-service-ca.crt\""
Jul 14 22:56:39.689027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.688989    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"rate-test-model-546296-kserve-self-signed-certs\""
Jul 14 22:56:39.689027 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.688946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-546296\"/\"kube-root-ca.crt\""
Jul 14 22:56:39.697746 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.697722    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"]
Jul 14 22:56:39.772369 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.772321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48430c34-6f7d-498f-9a12-0d72779a054c-tls-certs\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.772566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.772424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-home\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.772566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.772459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-dshm\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.772566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.772524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-tmp-dir\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.772566 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.772549    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whfx7\" (UniqueName: \"kubernetes.io/projected/48430c34-6f7d-498f-9a12-0d72779a054c-kube-api-access-whfx7\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.772741 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.772655    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-model-cache\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874032 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.873989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-home\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874032 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874037    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-dshm\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-tmp-dir\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whfx7\" (UniqueName: \"kubernetes.io/projected/48430c34-6f7d-498f-9a12-0d72779a054c-kube-api-access-whfx7\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-model-cache\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874319 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48430c34-6f7d-498f-9a12-0d72779a054c-tls-certs\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874535 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874428    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-home\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874535 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-tmp-dir\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.874651 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.874542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-model-cache\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.876374 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.876349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-dshm\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.876638 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.876621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48430c34-6f7d-498f-9a12-0d72779a054c-tls-certs\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.882867 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.882842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whfx7\" (UniqueName: \"kubernetes.io/projected/48430c34-6f7d-498f-9a12-0d72779a054c-kube-api-access-whfx7\") pod \"rate-test-model-546296-kserve-696c667bb7-mgdsq\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") " pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:39.994779 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:39.994685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:40.129530 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:40.129503    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"]
Jul 14 22:56:40.131448 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:56:40.131415    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48430c34_6f7d_498f_9a12_0d72779a054c.slice/crio-a49992b65024e6b224f7d1f38bd1b7e886a01aa4ccf8ef7bfa29e86b3ee8597c WatchSource:0}: Error finding container a49992b65024e6b224f7d1f38bd1b7e886a01aa4ccf8ef7bfa29e86b3ee8597c: Status 404 returned error can't find the container with id a49992b65024e6b224f7d1f38bd1b7e886a01aa4ccf8ef7bfa29e86b3ee8597c
Jul 14 22:56:40.870435 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:40.870392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq" event={"ID":"48430c34-6f7d-498f-9a12-0d72779a054c","Type":"ContainerStarted","Data":"74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551"}
Jul 14 22:56:40.870435 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:40.870441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq" event={"ID":"48430c34-6f7d-498f-9a12-0d72779a054c","Type":"ContainerStarted","Data":"a49992b65024e6b224f7d1f38bd1b7e886a01aa4ccf8ef7bfa29e86b3ee8597c"}
Jul 14 22:56:40.892663 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:40.892609    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq" podStartSLOduration=1.8925922769999999 podStartE2EDuration="1.892592277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:56:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:56:40.891611046 +0000 UTC m=+2841.777456622" watchObservedRunningTime="2026-07-14 22:56:40.892592277 +0000 UTC m=+2841.778437852"
Jul 14 22:56:49.995238 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:49.995206    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:49.995665 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:49.995262    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:50.007785 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:50.007756    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:56:50.931052 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:56:50.931025    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:57:29.422401 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:29.422309    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"]
Jul 14 22:57:30.292631 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:30.292596    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"]
Jul 14 22:57:44.442674 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.442634    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw" podUID="37069d38-1262-4e3c-92c5-c7da1081ae61" containerName="main" containerID="cri-o://351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447" gracePeriod=45
Jul 14 22:57:44.699653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.699589    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:57:44.799660 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.799626    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-dshm\") pod \"37069d38-1262-4e3c-92c5-c7da1081ae61\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") "
Jul 14 22:57:44.799856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.799686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37069d38-1262-4e3c-92c5-c7da1081ae61-tls-certs\") pod \"37069d38-1262-4e3c-92c5-c7da1081ae61\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") "
Jul 14 22:57:44.799856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.799739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-home\") pod \"37069d38-1262-4e3c-92c5-c7da1081ae61\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") "
Jul 14 22:57:44.799856 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.799764    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-tmp-dir\") pod \"37069d38-1262-4e3c-92c5-c7da1081ae61\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") "
Jul 14 22:57:44.800131 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.799912    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-model-cache\") pod \"37069d38-1262-4e3c-92c5-c7da1081ae61\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") "
Jul 14 22:57:44.800131 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.799956    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhl9s\" (UniqueName: \"kubernetes.io/projected/37069d38-1262-4e3c-92c5-c7da1081ae61-kube-api-access-qhl9s\") pod \"37069d38-1262-4e3c-92c5-c7da1081ae61\" (UID: \"37069d38-1262-4e3c-92c5-c7da1081ae61\") "
Jul 14 22:57:44.800131 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.800021    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "37069d38-1262-4e3c-92c5-c7da1081ae61" (UID: "37069d38-1262-4e3c-92c5-c7da1081ae61"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:44.800131 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.800032    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-home" (OuterVolumeSpecName: "home") pod "37069d38-1262-4e3c-92c5-c7da1081ae61" (UID: "37069d38-1262-4e3c-92c5-c7da1081ae61"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:44.800361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.800195    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-model-cache" (OuterVolumeSpecName: "model-cache") pod "37069d38-1262-4e3c-92c5-c7da1081ae61" (UID: "37069d38-1262-4e3c-92c5-c7da1081ae61"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:44.800361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.800339    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:44.800361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.800353    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:44.800361 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.800362    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:44.801936 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.801914    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-dshm" (OuterVolumeSpecName: "dshm") pod "37069d38-1262-4e3c-92c5-c7da1081ae61" (UID: "37069d38-1262-4e3c-92c5-c7da1081ae61"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:44.801936 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.801931    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37069d38-1262-4e3c-92c5-c7da1081ae61-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "37069d38-1262-4e3c-92c5-c7da1081ae61" (UID: "37069d38-1262-4e3c-92c5-c7da1081ae61"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:57:44.802071 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.802025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37069d38-1262-4e3c-92c5-c7da1081ae61-kube-api-access-qhl9s" (OuterVolumeSpecName: "kube-api-access-qhl9s") pod "37069d38-1262-4e3c-92c5-c7da1081ae61" (UID: "37069d38-1262-4e3c-92c5-c7da1081ae61"). InnerVolumeSpecName "kube-api-access-qhl9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:57:44.901895 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.901862    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhl9s\" (UniqueName: \"kubernetes.io/projected/37069d38-1262-4e3c-92c5-c7da1081ae61-kube-api-access-qhl9s\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:44.901895 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.901895    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/37069d38-1262-4e3c-92c5-c7da1081ae61-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:44.902137 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:44.901909    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37069d38-1262-4e3c-92c5-c7da1081ae61-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:45.176657 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.176622    2580 generic.go:356] "Generic (PLEG): container finished" podID="37069d38-1262-4e3c-92c5-c7da1081ae61" containerID="351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447" exitCode=0
Jul 14 22:57:45.176862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.176668    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"
Jul 14 22:57:45.176862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.176670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw" event={"ID":"37069d38-1262-4e3c-92c5-c7da1081ae61","Type":"ContainerDied","Data":"351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447"}
Jul 14 22:57:45.176862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.176790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw" event={"ID":"37069d38-1262-4e3c-92c5-c7da1081ae61","Type":"ContainerDied","Data":"41b1925d3fdefc4639fd7090afb1721b63e3159ae0e7ed4e920d49fa527707a7"}
Jul 14 22:57:45.176862 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.176813    2580 scope.go:117] "RemoveContainer" containerID="351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447"
Jul 14 22:57:45.186228 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.186204    2580 scope.go:117] "RemoveContainer" containerID="351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447"
Jul 14 22:57:45.186549 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:57:45.186526    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447\": container with ID starting with 351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447 not found: ID does not exist" containerID="351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447"
Jul 14 22:57:45.186648 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.186556    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447"} err="failed to get container status \"351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447\": rpc error: code = NotFound desc = could not find container \"351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447\": container with ID starting with 351e07aba4cdea696f7c46597c5f841550ec59f77a45a1522f6fd72cbd2db447 not found: ID does not exist"
Jul 14 22:57:45.201286 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.201258    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"]
Jul 14 22:57:45.202968 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.202945    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7ab9a6/rate-test-model-7ab9a6-kserve-5b68bd8755-qlgjw"]
Jul 14 22:57:45.313364 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.313325    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq" podUID="48430c34-6f7d-498f-9a12-0d72779a054c" containerName="main" containerID="cri-o://74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551" gracePeriod=45
Jul 14 22:57:45.571624 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.571603    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:57:45.602746 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.602700    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37069d38-1262-4e3c-92c5-c7da1081ae61" path="/var/lib/kubelet/pods/37069d38-1262-4e3c-92c5-c7da1081ae61/volumes"
Jul 14 22:57:45.709003 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.708903    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-model-cache\") pod \"48430c34-6f7d-498f-9a12-0d72779a054c\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") "
Jul 14 22:57:45.709196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709001    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-dshm\") pod \"48430c34-6f7d-498f-9a12-0d72779a054c\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") "
Jul 14 22:57:45.709196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709049    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whfx7\" (UniqueName: \"kubernetes.io/projected/48430c34-6f7d-498f-9a12-0d72779a054c-kube-api-access-whfx7\") pod \"48430c34-6f7d-498f-9a12-0d72779a054c\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") "
Jul 14 22:57:45.709196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-tmp-dir\") pod \"48430c34-6f7d-498f-9a12-0d72779a054c\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") "
Jul 14 22:57:45.709196 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709182    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48430c34-6f7d-498f-9a12-0d72779a054c-tls-certs\") pod \"48430c34-6f7d-498f-9a12-0d72779a054c\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") "
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-home\") pod \"48430c34-6f7d-498f-9a12-0d72779a054c\" (UID: \"48430c34-6f7d-498f-9a12-0d72779a054c\") "
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709256    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-model-cache" (OuterVolumeSpecName: "model-cache") pod "48430c34-6f7d-498f-9a12-0d72779a054c" (UID: "48430c34-6f7d-498f-9a12-0d72779a054c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709399    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48430c34-6f7d-498f-9a12-0d72779a054c" (UID: "48430c34-6f7d-498f-9a12-0d72779a054c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709507    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-home" (OuterVolumeSpecName: "home") pod "48430c34-6f7d-498f-9a12-0d72779a054c" (UID: "48430c34-6f7d-498f-9a12-0d72779a054c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709615    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-home\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709632    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-model-cache\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:45.709888 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.709647    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-tmp-dir\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:45.711313 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.711287    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48430c34-6f7d-498f-9a12-0d72779a054c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48430c34-6f7d-498f-9a12-0d72779a054c" (UID: "48430c34-6f7d-498f-9a12-0d72779a054c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:57:45.711420 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.711320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-dshm" (OuterVolumeSpecName: "dshm") pod "48430c34-6f7d-498f-9a12-0d72779a054c" (UID: "48430c34-6f7d-498f-9a12-0d72779a054c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:57:45.711420 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.711358    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48430c34-6f7d-498f-9a12-0d72779a054c-kube-api-access-whfx7" (OuterVolumeSpecName: "kube-api-access-whfx7") pod "48430c34-6f7d-498f-9a12-0d72779a054c" (UID: "48430c34-6f7d-498f-9a12-0d72779a054c"). InnerVolumeSpecName "kube-api-access-whfx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:57:45.810309 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.810261    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48430c34-6f7d-498f-9a12-0d72779a054c-tls-certs\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:45.810309 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.810303    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48430c34-6f7d-498f-9a12-0d72779a054c-dshm\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:45.810309 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:45.810312    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whfx7\" (UniqueName: \"kubernetes.io/projected/48430c34-6f7d-498f-9a12-0d72779a054c-kube-api-access-whfx7\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:46.037924 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.037890    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7720-140380355505728"]
Jul 14 22:57:46.038318 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.038302    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37069d38-1262-4e3c-92c5-c7da1081ae61" containerName="main"
Jul 14 22:57:46.038370 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.038321    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="37069d38-1262-4e3c-92c5-c7da1081ae61" containerName="main"
Jul 14 22:57:46.038370 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.038332    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48430c34-6f7d-498f-9a12-0d72779a054c" containerName="main"
Jul 14 22:57:46.038370 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.038337    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="48430c34-6f7d-498f-9a12-0d72779a054c" containerName="main"
Jul 14 22:57:46.038472 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.038425    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="37069d38-1262-4e3c-92c5-c7da1081ae61" containerName="main"
Jul 14 22:57:46.038472 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.038437    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="48430c34-6f7d-498f-9a12-0d72779a054c" containerName="main"
Jul 14 22:57:46.041664 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.041649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:46.045609 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.045585    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-784kt\""
Jul 14 22:57:46.050120 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.050079    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-140380355505728"]
Jul 14 22:57:46.113885 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.113849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88v96\" (UniqueName: \"kubernetes.io/projected/5a8ef807-3330-4407-b68f-e246cff8de1c-kube-api-access-88v96\") pod \"test-curl-7720-140380355505728\" (UID: \"5a8ef807-3330-4407-b68f-e246cff8de1c\") " pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:46.183615 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.183579    2580 generic.go:356] "Generic (PLEG): container finished" podID="48430c34-6f7d-498f-9a12-0d72779a054c" containerID="74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551" exitCode=0
Jul 14 22:57:46.183822 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.183627    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"
Jul 14 22:57:46.183822 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.183646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq" event={"ID":"48430c34-6f7d-498f-9a12-0d72779a054c","Type":"ContainerDied","Data":"74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551"}
Jul 14 22:57:46.183822 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.183683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq" event={"ID":"48430c34-6f7d-498f-9a12-0d72779a054c","Type":"ContainerDied","Data":"a49992b65024e6b224f7d1f38bd1b7e886a01aa4ccf8ef7bfa29e86b3ee8597c"}
Jul 14 22:57:46.183822 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.183704    2580 scope.go:117] "RemoveContainer" containerID="74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551"
Jul 14 22:57:46.200157 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.200137    2580 scope.go:117] "RemoveContainer" containerID="74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551"
Jul 14 22:57:46.200427 ip-10-0-143-117 kubenswrapper[2580]: E0714 22:57:46.200404    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551\": container with ID starting with 74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551 not found: ID does not exist" containerID="74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551"
Jul 14 22:57:46.200512 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.200432    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551"} err="failed to get container status \"74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551\": rpc error: code = NotFound desc = could not find container \"74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551\": container with ID starting with 74222fc22f30c7fbbed535a70cc3f3fb4c99e570ea30666904e4a7f7a0ad3551 not found: ID does not exist"
Jul 14 22:57:46.215144 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.214988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88v96\" (UniqueName: \"kubernetes.io/projected/5a8ef807-3330-4407-b68f-e246cff8de1c-kube-api-access-88v96\") pod \"test-curl-7720-140380355505728\" (UID: \"5a8ef807-3330-4407-b68f-e246cff8de1c\") " pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:46.216929 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.216906    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"]
Jul 14 22:57:46.225394 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.225369    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-546296/rate-test-model-546296-kserve-696c667bb7-mgdsq"]
Jul 14 22:57:46.226615 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.226595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88v96\" (UniqueName: \"kubernetes.io/projected/5a8ef807-3330-4407-b68f-e246cff8de1c-kube-api-access-88v96\") pod \"test-curl-7720-140380355505728\" (UID: \"5a8ef807-3330-4407-b68f-e246cff8de1c\") " pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:46.353006 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.352919    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:46.494450 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.494416    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7720-140380355505728"]
Jul 14 22:57:46.499995 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:57:46.499967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a8ef807_3330_4407_b68f_e246cff8de1c.slice/crio-4bc87d558d32c6ad41e060a89abbff3b1217d59d4cfa8324a63ddf7c3a70ca8d WatchSource:0}: Error finding container 4bc87d558d32c6ad41e060a89abbff3b1217d59d4cfa8324a63ddf7c3a70ca8d: Status 404 returned error can't find the container with id 4bc87d558d32c6ad41e060a89abbff3b1217d59d4cfa8324a63ddf7c3a70ca8d
Jul 14 22:57:46.501465 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:46.501443    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:57:47.190827 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:47.190794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-140380355505728" event={"ID":"5a8ef807-3330-4407-b68f-e246cff8de1c","Type":"ContainerStarted","Data":"4bc87d558d32c6ad41e060a89abbff3b1217d59d4cfa8324a63ddf7c3a70ca8d"}
Jul 14 22:57:47.603868 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:47.603829    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48430c34-6f7d-498f-9a12-0d72779a054c" path="/var/lib/kubelet/pods/48430c34-6f7d-498f-9a12-0d72779a054c/volumes"
Jul 14 22:57:48.197376 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:48.197342    2580 generic.go:356] "Generic (PLEG): container finished" podID="5a8ef807-3330-4407-b68f-e246cff8de1c" containerID="1a3984db654434f19d5352c6ca08515498fc4f866a3acdb9ab1b18bfab014a25" exitCode=0
Jul 14 22:57:48.197738 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:48.197409    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7720-140380355505728" event={"ID":"5a8ef807-3330-4407-b68f-e246cff8de1c","Type":"ContainerDied","Data":"1a3984db654434f19d5352c6ca08515498fc4f866a3acdb9ab1b18bfab014a25"}
Jul 14 22:57:49.338988 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.335690    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:49.393773 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.393738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7720-140380355505728_5a8ef807-3330-4407-b68f-e246cff8de1c/test-curl-7720-140380355505728/0.log"
Jul 14 22:57:49.426592 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.426549    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7720-140380355505728"]
Jul 14 22:57:49.429089 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.429057    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7720-140380355505728"]
Jul 14 22:57:49.445770 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.445743    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88v96\" (UniqueName: \"kubernetes.io/projected/5a8ef807-3330-4407-b68f-e246cff8de1c-kube-api-access-88v96\") pod \"5a8ef807-3330-4407-b68f-e246cff8de1c\" (UID: \"5a8ef807-3330-4407-b68f-e246cff8de1c\") "
Jul 14 22:57:49.447945 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.447914    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8ef807-3330-4407-b68f-e246cff8de1c-kube-api-access-88v96" (OuterVolumeSpecName: "kube-api-access-88v96") pod "5a8ef807-3330-4407-b68f-e246cff8de1c" (UID: "5a8ef807-3330-4407-b68f-e246cff8de1c"). InnerVolumeSpecName "kube-api-access-88v96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:57:49.547390 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.547348    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88v96\" (UniqueName: \"kubernetes.io/projected/5a8ef807-3330-4407-b68f-e246cff8de1c-kube-api-access-88v96\") on node \"ip-10-0-143-117.ec2.internal\" DevicePath \"\""
Jul 14 22:57:49.602488 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:49.602452    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8ef807-3330-4407-b68f-e246cff8de1c" path="/var/lib/kubelet/pods/5a8ef807-3330-4407-b68f-e246cff8de1c/volumes"
Jul 14 22:57:50.209642 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:50.209612    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7720-140380355505728"
Jul 14 22:57:50.209829 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:57:50.209680    2580 scope.go:117] "RemoveContainer" containerID="1a3984db654434f19d5352c6ca08515498fc4f866a3acdb9ab1b18bfab014a25"
Jul 14 22:58:47.800167 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:47.800060    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-p5vwz_86156926-0af8-4e53-ae27-de6cc9f11687/manager/0.log"
Jul 14 22:58:47.936001 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:47.935970    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5446c8b597-js2xc_6a822d58-eb7a-4d37-b8cd-0043d6b5207f/manager/0.log"
Jul 14 22:58:48.196452 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:48.196363    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mv56q_d07860df-2541-417b-b107-cc2e0aef1ce8/manager/0.log"
Jul 14 22:58:48.464278 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:48.464198    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c5998494-l2xww_e2bdee43-9359-4fd7-9b12-008715f35cb1/copy-manifests/0.log"
Jul 14 22:58:48.472926 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:48.472895    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c5998494-l2xww_e2bdee43-9359-4fd7-9b12-008715f35cb1/manager/0.log"
Jul 14 22:58:50.848470 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:50.848435    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mxkpg_28e77bbc-1b0f-49f2-9e28-5ae5ce417e52/postgres/0.log"
Jul 14 22:58:52.632588 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:52.632555    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8q6cj_e60c1b76-4a53-4279-9795-15b0aec484ab/kuadrant-console-plugin/0.log"
Jul 14 22:58:53.912187 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:53.912155    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ddb5db5fc-rqdfr_bcacde91-8a45-4c37-8d7d-be148ae4e1e1/kube-auth-proxy/0.log"
Jul 14 22:58:54.165712 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:54.165626    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-fkhks_f5fac104-5517-4ad9-a091-a186aafc2d92/payload-pre-processing/0.log"
Jul 14 22:58:54.292894 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:54.292867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ncdgq_c36be2b9-d826-4905-8d53-77d2ca3c45e3/payload-processing/0.log"
Jul 14 22:58:54.784077 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:54.784049    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jljn2_00a51bae-bace-44ad-9ac1-6721905228bf/main/0.log"
Jul 14 22:58:55.028675 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:55.028645    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-zpbr8_5feeb23f-ba95-426b-8fef-f791ea8a2836/main/0.log"
Jul 14 22:58:55.393270 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:58:55.393233    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pf4k9_33c8600e-67c7-4b75-ae9e-83fb1de8e121/main/0.log"
Jul 14 22:59:02.322978 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:02.322945    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rpzvt_951fc85f-b636-49b9-940f-96604cd2a47e/global-pull-secret-syncer/0.log"
Jul 14 22:59:02.428813 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:02.428778    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tgbvb_e4282920-0536-4ca7-82b0-cf9816fbe7c3/konnectivity-agent/0.log"
Jul 14 22:59:02.544886 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:02.544858    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-117.ec2.internal_55527337be11eb05926e7cca822eb38f/haproxy/0.log"
Jul 14 22:59:07.330535 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:07.330503    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8q6cj_e60c1b76-4a53-4279-9795-15b0aec484ab/kuadrant-console-plugin/0.log"
Jul 14 22:59:09.191227 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.191193    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/alertmanager/0.log"
Jul 14 22:59:09.215870 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.215840    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/config-reloader/0.log"
Jul 14 22:59:09.240285 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.240253    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/kube-rbac-proxy-web/0.log"
Jul 14 22:59:09.280080 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.280051    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/kube-rbac-proxy/0.log"
Jul 14 22:59:09.343729 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.343700    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/kube-rbac-proxy-metric/0.log"
Jul 14 22:59:09.369164 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.369078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/prom-label-proxy/0.log"
Jul 14 22:59:09.392653 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.392626    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_0604c386-1c02-4de1-8b05-6d46ea81eb87/init-config-reloader/0.log"
Jul 14 22:59:09.829460 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.829418    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bnscr_9e44fcd1-2f2f-4996-a283-105ca1a9841a/node-exporter/0.log"
Jul 14 22:59:09.855763 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.855722    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bnscr_9e44fcd1-2f2f-4996-a283-105ca1a9841a/kube-rbac-proxy/0.log"
Jul 14 22:59:09.881279 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:09.881251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bnscr_9e44fcd1-2f2f-4996-a283-105ca1a9841a/init-textfile/0.log"
Jul 14 22:59:10.018956 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.018933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/prometheus/0.log"
Jul 14 22:59:10.041706 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.041668    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/config-reloader/0.log"
Jul 14 22:59:10.065216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.065189    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/thanos-sidecar/0.log"
Jul 14 22:59:10.106798 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.106711    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/kube-rbac-proxy-web/0.log"
Jul 14 22:59:10.132439 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.132416    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/kube-rbac-proxy/0.log"
Jul 14 22:59:10.157055 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.157027    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/kube-rbac-proxy-thanos/0.log"
Jul 14 22:59:10.179509 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.179473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_84de2809-7e09-4f7d-819d-743129c347a7/init-config-reloader/0.log"
Jul 14 22:59:10.227800 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.227773    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-x4lkh_5cfaa129-e13e-41cf-b8d0-875f7226f3af/prometheus-operator/0.log"
Jul 14 22:59:10.248717 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.248681    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-x4lkh_5cfaa129-e13e-41cf-b8d0-875f7226f3af/kube-rbac-proxy/0.log"
Jul 14 22:59:10.282249 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.282227    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-xzvn8_72b047c9-aceb-47ed-9664-06d184199dd8/prometheus-operator-admission-webhook/0.log"
Jul 14 22:59:10.315284 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.315250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d85979485-bwvm9_752b042d-4925-4762-91f4-1ca883df869d/telemeter-client/0.log"
Jul 14 22:59:10.348955 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.348918    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d85979485-bwvm9_752b042d-4925-4762-91f4-1ca883df869d/reload/0.log"
Jul 14 22:59:10.371631 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.371552    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d85979485-bwvm9_752b042d-4925-4762-91f4-1ca883df869d/kube-rbac-proxy/0.log"
Jul 14 22:59:10.409914 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.409879    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d449f5c57-j2sqf_841f5f5e-81a0-446e-98e0-963c2d063053/thanos-query/0.log"
Jul 14 22:59:10.435979 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.435957    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d449f5c57-j2sqf_841f5f5e-81a0-446e-98e0-963c2d063053/kube-rbac-proxy-web/0.log"
Jul 14 22:59:10.460802 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.460784    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d449f5c57-j2sqf_841f5f5e-81a0-446e-98e0-963c2d063053/kube-rbac-proxy/0.log"
Jul 14 22:59:10.487002 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.486979    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d449f5c57-j2sqf_841f5f5e-81a0-446e-98e0-963c2d063053/prom-label-proxy/0.log"
Jul 14 22:59:10.514777 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.514755    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d449f5c57-j2sqf_841f5f5e-81a0-446e-98e0-963c2d063053/kube-rbac-proxy-rules/0.log"
Jul 14 22:59:10.550633 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:10.550605    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d449f5c57-j2sqf_841f5f5e-81a0-446e-98e0-963c2d063053/kube-rbac-proxy-metrics/0.log"
Jul 14 22:59:12.713562 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.713529    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"]
Jul 14 22:59:12.713957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.713910    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a8ef807-3330-4407-b68f-e246cff8de1c" containerName="test-curl-7720-140380355505728"
Jul 14 22:59:12.713957 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.713921    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a8ef807-3330-4407-b68f-e246cff8de1c" containerName="test-curl-7720-140380355505728"
Jul 14 22:59:12.714038 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.713972    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a8ef807-3330-4407-b68f-e246cff8de1c" containerName="test-curl-7720-140380355505728"
Jul 14 22:59:12.717109 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.717084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.720552 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.720527    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9nq2f\"/\"kube-root-ca.crt\""
Jul 14 22:59:12.721886 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.721865    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9nq2f\"/\"default-dockercfg-zqhpv\""
Jul 14 22:59:12.722011 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.721865    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9nq2f\"/\"openshift-service-ca.crt\""
Jul 14 22:59:12.726552 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.726525    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"]
Jul 14 22:59:12.821379 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.821345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-lib-modules\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.821572 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.821396    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4tj\" (UniqueName: \"kubernetes.io/projected/cea12852-f0b7-4e75-9832-2e77825cfd8b-kube-api-access-4t4tj\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.821572 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.821438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-proc\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.821572 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.821500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-sys\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.821706 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.821593    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-podres\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.922988 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.922947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4tj\" (UniqueName: \"kubernetes.io/projected/cea12852-f0b7-4e75-9832-2e77825cfd8b-kube-api-access-4t4tj\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.922999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-proc\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-sys\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-podres\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-proc\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923216 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-lib-modules\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923449 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-sys\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923449 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-lib-modules\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.923449 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.923278    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cea12852-f0b7-4e75-9832-2e77825cfd8b-podres\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:12.933460 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:12.933435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4tj\" (UniqueName: \"kubernetes.io/projected/cea12852-f0b7-4e75-9832-2e77825cfd8b-kube-api-access-4t4tj\") pod \"perf-node-gather-daemonset-cgqbr\" (UID: \"cea12852-f0b7-4e75-9832-2e77825cfd8b\") " pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:13.028122 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:13.028079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:13.161079 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:13.161050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"]
Jul 14 22:59:13.164657 ip-10-0-143-117 kubenswrapper[2580]: W0714 22:59:13.164625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcea12852_f0b7_4e75_9832_2e77825cfd8b.slice/crio-eb2aab8ce517716b86c9dfc1bbdbef7129b8e155aae4a3116d22ace0848d41be WatchSource:0}: Error finding container eb2aab8ce517716b86c9dfc1bbdbef7129b8e155aae4a3116d22ace0848d41be: Status 404 returned error can't find the container with id eb2aab8ce517716b86c9dfc1bbdbef7129b8e155aae4a3116d22ace0848d41be
Jul 14 22:59:13.614741 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:13.614705    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr" event={"ID":"cea12852-f0b7-4e75-9832-2e77825cfd8b","Type":"ContainerStarted","Data":"6f3803a4f0d8fa4bfd7425e906ed269d81f18229ff0b38846e62b8b5d0479e5d"}
Jul 14 22:59:13.614924 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:13.614756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr" event={"ID":"cea12852-f0b7-4e75-9832-2e77825cfd8b","Type":"ContainerStarted","Data":"eb2aab8ce517716b86c9dfc1bbdbef7129b8e155aae4a3116d22ace0848d41be"}
Jul 14 22:59:13.614924 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:13.614815    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:13.633377 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:13.633324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr" podStartSLOduration=1.6333082509999999 podStartE2EDuration="1.633308251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:59:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:59:13.631251551 +0000 UTC m=+2994.517097129" watchObservedRunningTime="2026-07-14 22:59:13.633308251 +0000 UTC m=+2994.519153857"
Jul 14 22:59:14.932253 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:14.932223    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6pqlf_ffe4d33d-1455-4c57-b848-c4f997e252c1/dns/0.log"
Jul 14 22:59:14.955780 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:14.955753    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6pqlf_ffe4d33d-1455-4c57-b848-c4f997e252c1/kube-rbac-proxy/0.log"
Jul 14 22:59:15.058158 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:15.058126    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fs8sr_502320b9-2d9b-411d-bc07-408eadcb47cb/dns-node-resolver/0.log"
Jul 14 22:59:15.661635 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:15.661602    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2ncv7_b5884e1b-5322-4f0a-8f7e-68993c54be15/node-ca/0.log"
Jul 14 22:59:16.969043 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:16.969013    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ddb5db5fc-rqdfr_bcacde91-8a45-4c37-8d7d-be148ae4e1e1/kube-auth-proxy/0.log"
Jul 14 22:59:17.080392 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:17.080365    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-fkhks_f5fac104-5517-4ad9-a091-a186aafc2d92/payload-pre-processing/0.log"
Jul 14 22:59:17.124161 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:17.124130    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ncdgq_c36be2b9-d826-4905-8d53-77d2ca3c45e3/payload-processing/0.log"
Jul 14 22:59:17.925807 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:17.925781    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cv9rh_af50125a-7868-4c53-9c40-307159cf7b1c/serve-healthcheck-canary/0.log"
Jul 14 22:59:18.558146 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:18.558117    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hpxdq_51ef2a3c-9a27-423c-9e89-82d015e195f2/kube-rbac-proxy/0.log"
Jul 14 22:59:18.578448 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:18.578414    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hpxdq_51ef2a3c-9a27-423c-9e89-82d015e195f2/exporter/0.log"
Jul 14 22:59:18.603080 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:18.603051    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hpxdq_51ef2a3c-9a27-423c-9e89-82d015e195f2/extractor/0.log"
Jul 14 22:59:19.629380 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:19.629353    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9nq2f/perf-node-gather-daemonset-cgqbr"
Jul 14 22:59:19.771604 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:19.771575    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:59:19.779259 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:19.779237    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-85x5g_7d0f8817-5cd4-4623-981d-8fea3863face/ovn-acl-logging/0.log"
Jul 14 22:59:21.052287 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:21.052251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-p5vwz_86156926-0af8-4e53-ae27-de6cc9f11687/manager/0.log"
Jul 14 22:59:21.227231 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:21.227197    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5446c8b597-js2xc_6a822d58-eb7a-4d37-b8cd-0043d6b5207f/manager/0.log"
Jul 14 22:59:21.362763 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:21.362676    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-mv56q_d07860df-2541-417b-b107-cc2e0aef1ce8/manager/0.log"
Jul 14 22:59:21.531689 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:21.531657    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c5998494-l2xww_e2bdee43-9359-4fd7-9b12-008715f35cb1/manager/0.log"
Jul 14 22:59:21.553744 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:21.553721    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86c5998494-l2xww_e2bdee43-9359-4fd7-9b12-008715f35cb1/copy-manifests/0.log"
Jul 14 22:59:22.923172 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:22.923140    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ddb5db5fc-rqdfr_bcacde91-8a45-4c37-8d7d-be148ae4e1e1/kube-auth-proxy/0.log"
Jul 14 22:59:22.967638 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:22.967605    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-66545d98-7dg8n_6597263d-7a30-4b6e-8acc-1bb93ffad911/manager/0.log"
Jul 14 22:59:23.005781 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:23.005756    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-fkhks_f5fac104-5517-4ad9-a091-a186aafc2d92/payload-pre-processing/0.log"
Jul 14 22:59:23.036493 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:23.036455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ncdgq_c36be2b9-d826-4905-8d53-77d2ca3c45e3/payload-processing/0.log"
Jul 14 22:59:24.604418 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:24.604342    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5ddb5db5fc-rqdfr_bcacde91-8a45-4c37-8d7d-be148ae4e1e1/kube-auth-proxy/0.log"
Jul 14 22:59:24.696967 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:24.696935    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-fkhks_f5fac104-5517-4ad9-a091-a186aafc2d92/payload-pre-processing/0.log"
Jul 14 22:59:24.727718 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:24.727691    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-ncdgq_c36be2b9-d826-4905-8d53-77d2ca3c45e3/payload-processing/0.log"
Jul 14 22:59:29.880340 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:29.880315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/kube-multus-additional-cni-plugins/0.log"
Jul 14 22:59:29.903962 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:29.903936    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/egress-router-binary-copy/0.log"
Jul 14 22:59:29.924937 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:29.924916    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/cni-plugins/0.log"
Jul 14 22:59:29.947404 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:29.947378    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/bond-cni-plugin/0.log"
Jul 14 22:59:29.971774 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:29.971750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/routeoverride-cni/0.log"
Jul 14 22:59:29.992423 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:29.992395    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/whereabouts-cni-bincopy/0.log"
Jul 14 22:59:30.025985 ip-10-0-143-117 kubenswrapper[2580]: I0714 22:59:30.025957    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkfmk_4191143e-ef8a-45bf-b6f2-900fcd15b83e/whereabouts-cni/0.log"