Jul 27 20:57:36.578489 ip-10-0-140-33 systemd[1]: Starting Kubernetes Kubelet...
Jul 27 20:57:36.963976 ip-10-0-140-33 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 20:57:36.963976 ip-10-0-140-33 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 27 20:57:36.963976 ip-10-0-140-33 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 20:57:36.963976 ip-10-0-140-33 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 27 20:57:36.963976 ip-10-0-140-33 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 27 20:57:36.964628 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.963960    2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 27 20:57:36.972683 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972656    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 20:57:36.972683 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972680    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 20:57:36.972683 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972685    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 20:57:36.972683 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972688    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 20:57:36.972683 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972691    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972694    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972697    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972699    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972702    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972705    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972709    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972713    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972716    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972719    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972722    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972724    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972728    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972732    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972734    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972739    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972742    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972745    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972747    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972750    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 20:57:36.972880 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972752    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972755    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972758    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972760    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972763    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972765    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972768    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972771    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972773    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972775    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972778    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972780    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972783    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972786    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972789    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972791    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972794    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972796    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972799    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972802    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 20:57:36.973394 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972804    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972807    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972809    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972812    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972814    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972821    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972823    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972826    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972829    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972831    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972834    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972837    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972840    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972843    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972845    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972848    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972850    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972853    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972855    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 20:57:36.973902 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972857    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972860    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972862    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972865    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972867    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972870    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972873    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972875    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972877    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972880    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972883    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972885    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972888    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972890    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972894    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972896    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972899    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972901    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972903    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972906    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 20:57:36.974370 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972909    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972911    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972916    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972920    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972923    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972926    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972929    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972932    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972935    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972937    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972940    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972942    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972945    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972947    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972950    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.972952    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973664    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973671    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973675    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 20:57:36.974834 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973679    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973683    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973686    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973689    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973691    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973694    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973697    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973699    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973702    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973704    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973707    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973710    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973712    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973714    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973717    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973720    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973722    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973725    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973727    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973730    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 20:57:36.975335 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973733    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973735    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973738    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973740    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973743    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973745    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973748    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973751    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973753    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973755    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973758    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973761    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973764    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973766    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973769    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973771    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973774    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973776    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973779    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973782    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 20:57:36.975827 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973784    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973787    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973789    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973791    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973794    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973796    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973798    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973802    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973804    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973807    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973810    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973812    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973815    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973817    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973820    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973822    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973824    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973827    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973829    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 20:57:36.976364 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973832    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973834    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973836    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973840    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973842    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973845    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973847    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973850    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973852    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973854    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973857    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973859    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973862    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973866    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973868    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973871    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973873    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973875    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973878    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973881    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 20:57:36.976820 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973883    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973886    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973890    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973893    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973896    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973899    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973902    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973905    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973907    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973910    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973913    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973915    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973918    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973920    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973923    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973925    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.973928    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974298    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974303    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974306    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 20:57:36.977323 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974309    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974311    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974314    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974317    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974319    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974322    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974324    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974327    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974329    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974332    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974334    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974337    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974339    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974342    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974344    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974347    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974350    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974352    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974356    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 20:57:36.977828 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974360    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974363    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974366    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974369    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974372    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974374    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974377    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974380    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974383    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974386    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974388    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974391    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974393    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974396    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974398    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974401    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974403    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974405    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974408    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974410    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 20:57:36.978308 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974412    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974415    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974417    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974420    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974423    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974427    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974430    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974433    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974436    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974439    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974441    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974444    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974446    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974448    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974451    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974453    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974456    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974458    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974460    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 20:57:36.978795 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974463    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974466    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974468    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974470    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974473    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974475    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974478    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974480    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974483    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974486    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974489    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974491    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974493    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974496    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974498    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974501    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974503    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974505    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974508    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974510    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 20:57:36.979254 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974513    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974516    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974518    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974521    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974523    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974526    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974528    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974530    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974533    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974535    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974537    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974540    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974543    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974545    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974547    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974550    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974554    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.974556    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974623    2552 flags.go:64] FLAG: --address="0.0.0.0"
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974629    2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974635    2552 flags.go:64] FLAG: --anonymous-auth="true"
Jul 27 20:57:36.979751 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974639    2552 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974644    2552 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974647    2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974650    2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974654    2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974657    2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974660    2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974663    2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974666    2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974669    2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974672    2552 flags.go:64] FLAG: --cgroup-root=""
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974675    2552 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974678    2552 flags.go:64] FLAG: --client-ca-file=""
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974682    2552 flags.go:64] FLAG: --cloud-provider="external"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974684    2552 flags.go:64] FLAG: --cluster-dns="[]"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974689    2552 flags.go:64] FLAG: --cluster-domain=""
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974691    2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974694    2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974697    2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974700    2552 flags.go:64] FLAG: --container-log-max-files="5"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974704    2552 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974707    2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974710    2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 27 20:57:36.980265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974713    2552 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974716    2552 flags.go:64] FLAG: --contention-profiling="false"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974719    2552 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974722    2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974726    2552 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974729    2552 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974732    2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974735    2552 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974738    2552 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974741    2552 flags.go:64] FLAG: --enable-load-reader="false"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974743    2552 flags.go:64] FLAG: --enable-server="true"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974746    2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974750    2552 flags.go:64] FLAG: --event-burst="100"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974753    2552 flags.go:64] FLAG: --event-qps="50"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974756    2552 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974758    2552 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974761    2552 flags.go:64] FLAG: --eviction-hard=""
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974765    2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974767    2552 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974770    2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974773    2552 flags.go:64] FLAG: --eviction-soft=""
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974776    2552 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974779    2552 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974782    2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974785    2552 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 27 20:57:36.980812 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974788    2552 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974790    2552 flags.go:64] FLAG: --fail-swap-on="true"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974793    2552 flags.go:64] FLAG: --feature-gates=""
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974796    2552 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974799    2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974802    2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974805    2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974808    2552 flags.go:64] FLAG: --healthz-port="10248"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974811    2552 flags.go:64] FLAG: --help="false"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974813    2552 flags.go:64] FLAG: --hostname-override="ip-10-0-140-33.ec2.internal"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974816    2552 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974819    2552 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974822    2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974825    2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974828    2552 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974830    2552 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974833    2552 flags.go:64] FLAG: --image-service-endpoint=""
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974836    2552 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974838    2552 flags.go:64] FLAG: --kube-api-burst="100"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974841    2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974845    2552 flags.go:64] FLAG: --kube-api-qps="50"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974847    2552 flags.go:64] FLAG: --kube-reserved=""
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974850    2552 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974853    2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 27 20:57:36.981442 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974855    2552 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974858    2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974861    2552 flags.go:64] FLAG: --lock-file=""
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974863    2552 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974866    2552 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974868    2552 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974873    2552 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974876    2552 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974879    2552 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974881    2552 flags.go:64] FLAG: --logging-format="text"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974884    2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974887    2552 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974889    2552 flags.go:64] FLAG: --manifest-url=""
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974892    2552 flags.go:64] FLAG: --manifest-url-header=""
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974898    2552 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974901    2552 flags.go:64] FLAG: --max-open-files="1000000"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974905    2552 flags.go:64] FLAG: --max-pods="110"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974908    2552 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974911    2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974913    2552 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974916    2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974919    2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974922    2552 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974924    2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974931    2552 flags.go:64] FLAG: --node-status-max-images="50"
Jul 27 20:57:36.982071 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974934    2552 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974936    2552 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974939    2552 flags.go:64] FLAG: --pod-cidr=""
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974942    2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974947    2552 flags.go:64] FLAG: --pod-manifest-path=""
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974949    2552 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974952    2552 flags.go:64] FLAG: --pods-per-core="0"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974955    2552 flags.go:64] FLAG: --port="10250"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974958    2552 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974961    2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-005df319611bc97c2"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974964    2552 flags.go:64] FLAG: --qos-reserved=""
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974967    2552 flags.go:64] FLAG: --read-only-port="10255"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974969    2552 flags.go:64] FLAG: --register-node="true"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974972    2552 flags.go:64] FLAG: --register-with-taints=""
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974975    2552 flags.go:64] FLAG: --registry-burst="10"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974979    2552 flags.go:64] FLAG: --registry-qps="5"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974981    2552 flags.go:64] FLAG: --reserved-cpus=""
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974984    2552 flags.go:64] FLAG: --reserved-memory=""
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974987    2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974990    2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974993    2552 flags.go:64] FLAG: --rotate-certificates="false"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.974997    2552 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975000    2552 flags.go:64] FLAG: --runonce="false"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975002    2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975005    2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 27 20:57:36.982735 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975008    2552 flags.go:64] FLAG: --seccomp-default="false"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975011    2552 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975014    2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975017    2552 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975019    2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975022    2552 flags.go:64] FLAG: --storage-driver-password="root"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975025    2552 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975027    2552 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975030    2552 flags.go:64] FLAG: --storage-driver-user="root"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975033    2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975036    2552 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975038    2552 flags.go:64] FLAG: --system-cgroups=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975041    2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975046    2552 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975049    2552 flags.go:64] FLAG: --tls-cert-file=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975051    2552 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975055    2552 flags.go:64] FLAG: --tls-min-version=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975058    2552 flags.go:64] FLAG: --tls-private-key-file=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975060    2552 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975063    2552 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975066    2552 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975069    2552 flags.go:64] FLAG: --v="2"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975072    2552 flags.go:64] FLAG: --version="false"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975076    2552 flags.go:64] FLAG: --vmodule=""
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975080    2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 27 20:57:36.983360 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.975086    2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975193    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975196    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975200    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975203    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975207    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975210    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975213    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975216    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975219    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975238    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975241    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975244    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975246    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975249    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975251    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975254    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975257    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975260    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 20:57:36.984045 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975262    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975265    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975267    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975270    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975273    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975275    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975278    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975280    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975282    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975285    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975288    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975290    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975293    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975297    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975300    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975302    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975307    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975309    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975312    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975314    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975317    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 20:57:36.984533 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975319    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975321    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975324    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975326    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975329    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975331    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975333    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975336    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975338    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975341    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975344    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975347    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975349    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975352    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975354    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975357    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975359    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975361    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975364    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975367    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 20:57:36.985074 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975369    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975371    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975373    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975376    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975380    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975382    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975386    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975391    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975394    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975396    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975399    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975401    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975404    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975406    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975408    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975410    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975413    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975415    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975418    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 20:57:36.985577 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975420    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975422    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975425    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975427    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975430    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975433    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975435    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975437    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975440    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975442    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975445    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975447    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975449    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975453    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975475    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975479    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975482    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975486    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975489    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975492    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 20:57:36.986032 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.975496    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.976287    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.983465    2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.983481    2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.984351    2552 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.984359    2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984414    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984418    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984422    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984425    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984428    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984432    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984434    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984437    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984440    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 20:57:36.986523 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984442    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984445    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984448    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984450    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984453    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984455    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984458    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984460    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984463    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984465    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984468    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984470    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984473    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984477    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984480    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984483    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984487    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984490    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984493    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 20:57:36.986906 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984496    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984501    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984503    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984506    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984509    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984511    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984514    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984516    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984519    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984522    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984525    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984527    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984530    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984533    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984535    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984538    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984540    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984543    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984546    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984549    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 20:57:36.987374 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984551    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984554    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984557    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984559    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984562    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984564    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984566    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984569    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984572    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984575    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984578    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984580    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984583    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984586    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984589    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984591    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984594    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984596    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984598    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984601    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 20:57:36.987857 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984603    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984605    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984608    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984610    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984612    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984615    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984617    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984619    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984622    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984624    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984628    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984632    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984634    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984637    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984640    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984642    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984645    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984647    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984649    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984652    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 20:57:36.988389 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984654    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984657    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984659    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984662    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984665    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984667    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984670    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984673    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984675    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984677    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984680    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.984684    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984793    2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984797    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 27 20:57:36.988876 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984801    2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984803    2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984806    2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984808    2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984811    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984813    2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984816    2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984818    2552 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984821    2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984823    2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984826    2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984828    2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984831    2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984833    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984835    2552 feature_gate.go:328] unrecognized feature gate: Example
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984838    2552 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984841    2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984843    2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984845    2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984847    2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 27 20:57:36.989234 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984850    2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984853    2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984855    2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984857    2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984860    2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984862    2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984865    2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984868    2552 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984870    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984872    2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984875    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984877    2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984880    2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984884    2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984888    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984891    2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984894    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984897    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984899    2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 27 20:57:36.989725 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984902    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984905    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984907    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984910    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984912    2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984915    2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984917    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984919    2552 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984922    2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984924    2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984926    2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984929    2552 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984931    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984934    2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984936    2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984939    2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984941    2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984944    2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984946    2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984948    2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 27 20:57:36.990188 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984951    2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984953    2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984956    2552 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984958    2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984961    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984963    2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984965    2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984968    2552 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984970    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984972    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984975    2552 feature_gate.go:328] unrecognized feature gate: Example2
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984977    2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984979    2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984982    2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984984    2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984986    2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984989    2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984991    2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984994    2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984996    2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 27 20:57:36.990763 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.984998    2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985000    2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985003    2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985005    2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985007    2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985010    2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985012    2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985015    2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985017    2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985019    2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985022    2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985024    2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985026    2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985029    2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985031    2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985034    2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985036    2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 27 20:57:36.991243 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:36.985039    2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.985044    2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.985138    2552 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.987636    2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.989144    2552 server.go:1019] "Starting client certificate rotation"
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.989255    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.989969    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 27 20:57:36.991673 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.991663    2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 20:57:36.996545 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:36.996525    2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 27 20:57:37.005860 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.005845    2552 log.go:25] "Validated CRI v1 runtime API"
Jul 27 20:57:37.008153 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.008138    2552 log.go:25] "Validated CRI v1 image API"
Jul 27 20:57:37.008932 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.008916    2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 27 20:57:37.011162 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.011145    2552 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b25cba4d-9b7f-4a30-a7d5-4d4e88e7f352:/dev/nvme0n1p3 fca062be-0a67-42ca-bfc0-35cf69de4500:/dev/nvme0n1p4]
Jul 27 20:57:37.011212 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.011163    2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 27 20:57:37.015613 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.015518    2552 manager.go:216] Machine: {Timestamp:2026-07-27 20:57:37.015047407 +0000 UTC m=+0.318292163 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096716 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c61655e1df3e3e6f3db2028a4623b SystemUUID:ec2c6165-5e1d-f3e3-e6f3-db2028a4623b BootID:7bcbb8ae-8ceb-4938-a653-670cced5b3e7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:68:fe:cb:b7:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:68:fe:cb:b7:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:3d:10:63:11:fa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 27 20:57:37.015613 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.015610    2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 27 20:57:37.015713 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.015699    2552 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 27 20:57:37.016185 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.016167    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 20:57:37.019173 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.019150    2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 27 20:57:37.019317 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.019177    2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-33.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 27 20:57:37.019922 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.019912    2552 topology_manager.go:138] "Creating topology manager with none policy"
Jul 27 20:57:37.019955 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.019925    2552 container_manager_linux.go:309] "Creating device plugin manager"
Jul 27 20:57:37.019955 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.019932    2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 20:57:37.019955 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.019942    2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 27 20:57:37.021670 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.021660    2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 27 20:57:37.025242 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025216    2552 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 20:57:37.025347 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025337    2552 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 27 20:57:37.025388 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025364    2552 kubelet.go:486] "Attempting to sync node with API server"
Jul 27 20:57:37.025388 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025372    2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 27 20:57:37.025388 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025387    2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 27 20:57:37.025505 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025396    2552 kubelet.go:398] "Adding apiserver pod source"
Jul 27 20:57:37.025505 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025404    2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 27 20:57:37.025733 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025721    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 20:57:37.025766 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.025741    2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 27 20:57:37.027973 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.027957    2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 27 20:57:37.029743 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.029727    2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 27 20:57:37.029803 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.029758    2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 27 20:57:37.030618 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030607    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 27 20:57:37.030654 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030624    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 27 20:57:37.030654 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030634    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 27 20:57:37.030654 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030644    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 27 20:57:37.030654 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030654    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030660    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030669    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030674    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030680    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030685    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030698    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 27 20:57:37.030748 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.030704    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 27 20:57:37.031419 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.031408    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 27 20:57:37.031456 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.031420    2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 27 20:57:37.033799 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.033781    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6kmrp"
Jul 27 20:57:37.034367 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.034340    2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-33.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 27 20:57:37.034666 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.034648    2552 server.go:1268] "Started kubelet"
Jul 27 20:57:37.034767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.034712    2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 27 20:57:37.034955 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.034933    2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 27 20:57:37.035021 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.034974    2552 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 27 20:57:37.035147 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.035118    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 27 20:57:37.035399 ip-10-0-140-33 systemd[1]: Started Kubernetes Kubelet.
Jul 27 20:57:37.038596 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.038461    2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-33.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 27 20:57:37.039094 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.039074    2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 27 20:57:37.039883 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.039862    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6kmrp"
Jul 27 20:57:37.041727 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.041708    2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 27 20:57:37.041727 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.041718    2552 server.go:310] "Adding debug handlers to kubelet server"
Jul 27 20:57:37.043354 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.042360    2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-33.ec2.internal.18c6411422999fcf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-33.ec2.internal,UID:ip-10-0-140-33.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-33.ec2.internal,},FirstTimestamp:2026-07-27 20:57:37.034624975 +0000 UTC m=+0.337869731,LastTimestamp:2026-07-27 20:57:37.034624975 +0000 UTC m=+0.337869731,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-33.ec2.internal,}"
Jul 27 20:57:37.050495 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.050478    2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 27 20:57:37.050582 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.050486    2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 27 20:57:37.050907 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.050885    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.051334 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051319    2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 27 20:57:37.051334 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051320    2552 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 27 20:57:37.051456 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051344    2552 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 27 20:57:37.051456 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051436    2552 reconstruct.go:97] "Volume reconstruction finished"
Jul 27 20:57:37.051456 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051446    2552 reconciler.go:29] "Reconciler: start to sync state"
Jul 27 20:57:37.051632 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051585    2552 factory.go:55] Registering systemd factory
Jul 27 20:57:37.051632 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051598    2552 factory.go:223] Registration of the systemd container factory successfully
Jul 27 20:57:37.051799 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051780    2552 factory.go:153] Registering CRI-O factory
Jul 27 20:57:37.051799 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051795    2552 factory.go:223] Registration of the crio container factory successfully
Jul 27 20:57:37.051907 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051836    2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 27 20:57:37.051907 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051854    2552 factory.go:103] Registering Raw factory
Jul 27 20:57:37.051907 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.051867    2552 manager.go:1196] Started watching for new ooms in manager
Jul 27 20:57:37.052265 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.052252    2552 manager.go:349] Starting recovery of all containers
Jul 27 20:57:37.059854 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.059823    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 20:57:37.061883 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.061868    2552 manager.go:354] Recovery completed
Jul 27 20:57:37.062549 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.062530    2552 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-140-33.ec2.internal\" not found" node="ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.065645 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.065632    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.068148 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068128    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.068255 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068161    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.068255 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068175    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.068663 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068651    2552 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 27 20:57:37.068663 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068662    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.068745 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068661    2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 27 20:57:37.068745 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.068720    2552 state_mem.go:36] "Initialized new in-memory state store"
Jul 27 20:57:37.069358 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.069345    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.069416 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.069372    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.069416 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.069384    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.069908 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.069897    2552 policy_none.go:49] "None policy: Start"
Jul 27 20:57:37.069940 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.069913    2552 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 27 20:57:37.069940 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.069923    2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 27 20:57:37.070926 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.070916    2552 policy_none.go:47] "Start"
Jul 27 20:57:37.115251 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115215    2552 manager.go:337] "Starting Device Plugin manager"
Jul 27 20:57:37.115326 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.115270    2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 27 20:57:37.115326 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115283    2552 server.go:87] "Starting device plugin registration server"
Jul 27 20:57:37.115499 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115487    2552 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 27 20:57:37.115551 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115501    2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 27 20:57:37.115607 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115587    2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 27 20:57:37.115836 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115683    2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 27 20:57:37.115917 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.115839    2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 27 20:57:37.116090 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.116074    2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 27 20:57:37.116146 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.116119    2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.189729 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.189707    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 27 20:57:37.191062 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.191046    2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 27 20:57:37.191114 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.191094    2552 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 27 20:57:37.191154 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.191121    2552 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 27 20:57:37.191207 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.191155    2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 27 20:57:37.193712 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.193692    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 20:57:37.215754 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.215709    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.216713 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.216698    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.216767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.216727    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.216767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.216737    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.216767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.216758    2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.225025 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.225012    2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.225072 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.225031    2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-33.ec2.internal\": node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.237402 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.237387    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.291596 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.291574    2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"]
Jul 27 20:57:37.291664 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.291631    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.292441 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.292422    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.292509 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.292447    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.292509 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.292457    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.293783 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.293771    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.293912 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.293900    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.293954 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.293932    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.294493 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.294475    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.294576 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.294506    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.294576 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.294517    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.294576 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.294475    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.294671 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.294577    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.294671 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.294589    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.296097 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.296080    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.296172 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.296105    2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 27 20:57:37.296844 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.296829    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientMemory"
Jul 27 20:57:37.296913 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.296858    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasNoDiskPressure"
Jul 27 20:57:37.296913 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.296870    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeHasSufficientPID"
Jul 27 20:57:37.313715 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.313692    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-33.ec2.internal\" not found" node="ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.317634 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.317620    2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-33.ec2.internal\" not found" node="ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.337725 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.337704    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.352683 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.352664    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce115a00c211e7ec27fc8ba3d423f461-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal\" (UID: \"ce115a00c211e7ec27fc8ba3d423f461\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.352683 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.352684    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce115a00c211e7ec27fc8ba3d423f461-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal\" (UID: \"ce115a00c211e7ec27fc8ba3d423f461\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.352807 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.352701    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/673a856a2cbcee0e523f1fb250518aab-config\") pod \"kube-apiserver-proxy-ip-10-0-140-33.ec2.internal\" (UID: \"673a856a2cbcee0e523f1fb250518aab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.438662 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.438629    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.452906 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.452883    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce115a00c211e7ec27fc8ba3d423f461-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal\" (UID: \"ce115a00c211e7ec27fc8ba3d423f461\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.452986 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.452913    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce115a00c211e7ec27fc8ba3d423f461-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal\" (UID: \"ce115a00c211e7ec27fc8ba3d423f461\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.452986 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.452936    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/673a856a2cbcee0e523f1fb250518aab-config\") pod \"kube-apiserver-proxy-ip-10-0-140-33.ec2.internal\" (UID: \"673a856a2cbcee0e523f1fb250518aab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.452986 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.452981    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ce115a00c211e7ec27fc8ba3d423f461-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal\" (UID: \"ce115a00c211e7ec27fc8ba3d423f461\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.453100 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.452994    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce115a00c211e7ec27fc8ba3d423f461-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal\" (UID: \"ce115a00c211e7ec27fc8ba3d423f461\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.453100 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.452995    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/673a856a2cbcee0e523f1fb250518aab-config\") pod \"kube-apiserver-proxy-ip-10-0-140-33.ec2.internal\" (UID: \"673a856a2cbcee0e523f1fb250518aab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.539345 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.539287    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.616759 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.616732    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.621775 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.621676    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:37.640344 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.640321    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.740841 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.740814    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.841382 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.841328    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.941499 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:37.941480    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:37.988530 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.988510    2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 27 20:57:37.988895 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.988651    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 20:57:37.988895 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:37.988688    2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 27 20:57:38.041946 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:38.041925    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:38.046032 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.045994    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 20:52:37 +0000 UTC" deadline="2028-04-17 19:36:48.080418314 +0000 UTC"
Jul 27 20:57:38.046032 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.046031    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15118h39m10.03439025s"
Jul 27 20:57:38.050696 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.050680    2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 20:57:38.126148 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:38.126118    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673a856a2cbcee0e523f1fb250518aab.slice/crio-777075347cfc4df4758685d571f62feeb59152c883fd970e0bb26968561aa2c7 WatchSource:0}: Error finding container 777075347cfc4df4758685d571f62feeb59152c883fd970e0bb26968561aa2c7: Status 404 returned error can't find the container with id 777075347cfc4df4758685d571f62feeb59152c883fd970e0bb26968561aa2c7
Jul 27 20:57:38.126419 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:38.126399    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce115a00c211e7ec27fc8ba3d423f461.slice/crio-0fbc3ae0af0c3c7ae1020863e16eb15258e84d8103fcb29d30299acb1209fa41 WatchSource:0}: Error finding container 0fbc3ae0af0c3c7ae1020863e16eb15258e84d8103fcb29d30299acb1209fa41: Status 404 returned error can't find the container with id 0fbc3ae0af0c3c7ae1020863e16eb15258e84d8103fcb29d30299acb1209fa41
Jul 27 20:57:38.130501 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.130486    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 20:57:38.142826 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:38.142807    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:38.194361 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.194316    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal" event={"ID":"ce115a00c211e7ec27fc8ba3d423f461","Type":"ContainerStarted","Data":"0fbc3ae0af0c3c7ae1020863e16eb15258e84d8103fcb29d30299acb1209fa41"}
Jul 27 20:57:38.195363 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.195340    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal" event={"ID":"673a856a2cbcee0e523f1fb250518aab","Type":"ContainerStarted","Data":"777075347cfc4df4758685d571f62feeb59152c883fd970e0bb26968561aa2c7"}
Jul 27 20:57:38.246359 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:38.246343    2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-33.ec2.internal\" not found"
Jul 27 20:57:38.344101 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.344071    2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 20:57:38.351271 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.351249    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:38.359553 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.359536    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 20:57:38.360579 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.360567    2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal"
Jul 27 20:57:38.370095 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.370050    2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 27 20:57:38.823907 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.823820    2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 20:57:38.958538 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:38.958510    2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 27 20:57:39.025847 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.025816    2552 apiserver.go:52] "Watching apiserver"
Jul 27 20:57:39.033214 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.033189    2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 27 20:57:39.034550 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.034523    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-4x6mt","kube-system/konnectivity-agent-h7sbg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal","openshift-multus/multus-c7vfj","openshift-network-operator/iptables-alerter-tzsln","openshift-ovn-kubernetes/ovnkube-node-h6fb9","kube-system/global-pull-secret-syncer-dp2gb","kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8q2ls","openshift-dns/node-resolver-dc22d","openshift-image-registry/node-ca-qnt7r","openshift-multus/multus-additional-cni-plugins-w6zkl","openshift-multus/network-metrics-daemon-cgq4s"]
Jul 27 20:57:39.036145 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.036122    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.038598 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.038576    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.041568 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.041544    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 27 20:57:39.041806 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.041786    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.041950 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.041927    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.042795 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.042137    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f7jsp\""
Jul 27 20:57:39.042795 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.042156    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.042795 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.039811    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.043894 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.043185    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.044616 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.044597    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 27 20:57:39.044813 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.044782    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.044930 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.044871    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:39.046719 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.046683    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.047390 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.047304    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g2jnv\""
Jul 27 20:57:39.047390 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.047363    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 27 20:57:39.047927 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.047548    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 27 20:57:39.047927 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.047629    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.047927 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.047752    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.047927 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.047805    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9cjdn\""
Jul 27 20:57:39.049485 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.048850    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.049485 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.049051    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-j6nrj\""
Jul 27 20:57:39.049485 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.049220    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 27 20:57:39.050478 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.050033    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.050707 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.050690    2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 27 20:57:39.052020 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.051999    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xnc99\""
Jul 27 20:57:39.052173 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052162    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 27 20:57:39.052616 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052410    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.052616 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052420    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.052616 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052544    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.052855 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052634    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.052913 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052870    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g4dd7\""
Jul 27 20:57:39.052965 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.052947    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.054563 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.054261    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:39.054563 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.054375    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:39.055080 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.055064    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 27 20:57:39.056214 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.056033    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 27 20:57:39.056214 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.056072    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 27 20:57:39.056214 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.056087    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.056214 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.056090    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qkcn5\""
Jul 27 20:57:39.056214 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.056137    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 27 20:57:39.056525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.056507    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.057757 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.057740    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.060767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.059753    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.060767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.060122    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tlv5l\""
Jul 27 20:57:39.060767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.060157    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 27 20:57:39.060767 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.060669    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 27 20:57:39.062580 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062359    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rv7sd\""
Jul 27 20:57:39.062675 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062636    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-conf-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.062675 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062665    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-tmp-dir\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.062797 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062691    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-etc-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.062797 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062716    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-ovnkube-script-lib\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.062797 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062755    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-system-cni-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.062797 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062778    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-systemd-units\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062804    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062826    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-k8s-cni-cncf-io\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062851    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-netns\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062874    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-kubelet\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062899    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-daemon-config\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062946    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-var-lib-kubelet\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.063002 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.062970    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-host\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063008    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-env-overrides\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063047    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-registration-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063072    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-socket-dir-parent\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063099    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063108    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-multus-certs\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063133    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4aef69b4-c167-4b19-98a6-f97b615b3d25-host-slash\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063157    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-var-lib-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063204    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5cps\" (UniqueName: \"kubernetes.io/projected/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-kube-api-access-n5cps\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063231    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jtzv\" (UniqueName: \"kubernetes.io/projected/0d1c14b4-7aec-4c81-98c5-55841ddfa641-kube-api-access-2jtzv\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063255    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-cni-bin\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063362    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-systemd\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063380    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063392    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063409    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063419    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67a066b6-b4af-49df-bc8b-5ae94c66ac84-cni-binary-copy\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063487    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-os-release\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063516    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063579    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-etc-kubernetes\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063611    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptzp9\" (UniqueName: \"kubernetes.io/projected/4aef69b4-c167-4b19-98a6-f97b615b3d25-kube-api-access-ptzp9\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063630    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-hosts-file\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063656    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-lib-modules\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063673    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzcrr\" (UniqueName: \"kubernetes.io/projected/e821c049-0ef1-44d3-8875-fae86b286c7b-kube-api-access-gzcrr\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063689    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-log-socket\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063704    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-sys\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063734    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-tuned\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063750    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-ovnkube-config\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063767    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-cni-netd\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.063911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063782    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3e0279f-9590-4e8e-bac3-1042620ed555-ovn-node-metrics-cert\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063811    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cni-binary-copy\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063836    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-device-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063861    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063885    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-cni-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063958    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgj6h\" (UniqueName: \"kubernetes.io/projected/67a066b6-b4af-49df-bc8b-5ae94c66ac84-kube-api-access-pgj6h\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.063994    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e821c049-0ef1-44d3-8875-fae86b286c7b-tmp\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064019    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-kubelet\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064038    2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064044    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hbwz\" (UniqueName: \"kubernetes.io/projected/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-kube-api-access-9hbwz\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064179    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-kubernetes\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064196    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-systemd\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064215    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-os-release\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064230    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-hostroot\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064246    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-modprobe-d\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064281    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-run\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064333    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-slash\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.064619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064365    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064407    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-system-cni-dir\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064440    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064467    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-socket-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064493    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysconfig\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064520    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysctl-d\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064545    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-cni-bin\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064568    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-cnibin\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064594    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khw4k\" (UniqueName: \"kubernetes.io/projected/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-kube-api-access-khw4k\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064618    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-ovn\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064644    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-run-ovn-kubernetes\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064681    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dxhc\" (UniqueName: \"kubernetes.io/projected/c3e0279f-9590-4e8e-bac3-1042620ed555-kube-api-access-2dxhc\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064711    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064740    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064777    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-sys-fs\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064812    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4aef69b4-c167-4b19-98a6-f97b615b3d25-iptables-alerter-script\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.065473 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064840    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysctl-conf\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.066399 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064878    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-run-netns\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.066399 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064901    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-node-log\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.066399 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064923    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cnibin\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.066399 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.064947    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-cni-multus\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.068472 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.068449    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.068593 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.068534    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:39.084605 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.084585    2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tjs99"
Jul 27 20:57:39.092792 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.092764    2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tjs99"
Jul 27 20:57:39.152804 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.152779    2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 27 20:57:39.166171 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166147    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-os-release\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.166297 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166185    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-hostroot\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.166297 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166213    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-modprobe-d\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166291    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-run\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166301    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-os-release\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166291    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-hostroot\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166329    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-slash\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166357    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166373    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-modprobe-d\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166379 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166381    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-system-cni-dir\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166388    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-slash\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166410    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-run\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166414    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166428    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-system-cni-dir\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166435    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166455    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-socket-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166461    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166475    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysconfig\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166529    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysconfig\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166554    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysctl-d\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166581    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-cni-bin\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166596    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-socket-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.166622 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166606    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-cnibin\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166632    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khw4k\" (UniqueName: \"kubernetes.io/projected/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-kube-api-access-khw4k\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166649    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-cni-bin\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166658    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-ovn\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166665    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysctl-d\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166683    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-run-ovn-kubernetes\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166703    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-cnibin\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166710    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dxhc\" (UniqueName: \"kubernetes.io/projected/c3e0279f-9590-4e8e-bac3-1042620ed555-kube-api-access-2dxhc\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166737    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166746    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-ovn\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166764    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166789    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-sys-fs\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166816    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4aef69b4-c167-4b19-98a6-f97b615b3d25-iptables-alerter-script\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166838    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysctl-conf\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166861    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-run-netns\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166884    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-node-log\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166908    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-sys-fs\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.167053 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166918    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cnibin\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166949    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcpbn\" (UniqueName: \"kubernetes.io/projected/93ced636-3c9e-428d-80e8-acb5ea71c7d2-kube-api-access-zcpbn\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.166976    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-cni-multus\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167003    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-conf-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167028    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-tmp-dir\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167054    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-etc-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167080    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-ovnkube-script-lib\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167104    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-system-cni-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167150    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-systemd-units\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167176    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167201    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-k8s-cni-cncf-io\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167233    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-netns\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167259    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-kubelet\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167283    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-daemon-config\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167329    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-var-lib-kubelet\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167381    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167408    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cnibin\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.167761 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167387    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-etc-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167443    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-system-cni-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167468    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-k8s-cni-cncf-io\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167492    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-netns\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167387    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-cni-multus\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167002    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-run-ovn-kubernetes\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167538    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-conf-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167561    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-host\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167578    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-env-overrides\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167581    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167596    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-registration-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167628    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-socket-dir-parent\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167636    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-registration-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167632    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-tmp-dir\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167644    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167695    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-kubelet\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167686    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-socket-dir-parent\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.168525 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167742    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-var-lib-kubelet\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167756    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-multus-certs\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167784    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4aef69b4-c167-4b19-98a6-f97b615b3d25-host-slash\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167814    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-var-lib-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167840    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5cps\" (UniqueName: \"kubernetes.io/projected/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-kube-api-access-n5cps\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167867    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jtzv\" (UniqueName: \"kubernetes.io/projected/0d1c14b4-7aec-4c81-98c5-55841ddfa641-kube-api-access-2jtzv\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167879    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-sysctl-conf\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167896    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3851429a-b5b9-4a29-b124-f84984721273-kubelet-config\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167923    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167928    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4aef69b4-c167-4b19-98a6-f97b615b3d25-host-slash\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167948    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-cni-bin\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167970    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-run-multus-certs\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167972    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-systemd\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168002    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4aef69b4-c167-4b19-98a6-f97b615b3d25-iptables-alerter-script\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168012    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-run-systemd\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168014    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168022    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-ovnkube-script-lib\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169125 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168047    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.167496    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-systemd-units\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168049    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-var-lib-openvswitch\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168065    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-env-overrides\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168079    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-node-log\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168125    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-run-netns\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168126    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168159    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fcfc6382-3050-4547-a388-ff49bbb472c4-agent-certs\") pod \"konnectivity-agent-h7sbg\" (UID: \"fcfc6382-3050-4547-a388-ff49bbb472c4\") " pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168183    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-host\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168186    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93ced636-3c9e-428d-80e8-acb5ea71c7d2-serviceca\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168208    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-daemon-config\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168215    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67a066b6-b4af-49df-bc8b-5ae94c66ac84-cni-binary-copy\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168246    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-os-release\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168253    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-host-var-lib-cni-bin\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168312    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-os-release\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168319    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168363    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-etc-kubernetes\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168391    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptzp9\" (UniqueName: \"kubernetes.io/projected/4aef69b4-c167-4b19-98a6-f97b615b3d25-kube-api-access-ptzp9\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.169928 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.168396    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168416    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-hosts-file\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168439    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-lib-modules\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168466    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzcrr\" (UniqueName: \"kubernetes.io/projected/e821c049-0ef1-44d3-8875-fae86b286c7b-kube-api-access-gzcrr\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168490    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-log-socket\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168503    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-hosts-file\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.168515    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:39.668491323 +0000 UTC m=+2.971736089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168528    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-log-socket\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168549    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-sys\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168562    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-etc-kubernetes\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168577    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-tuned\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168603    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-ovnkube-config\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168625    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-cni-netd\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168649    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3e0279f-9590-4e8e-bac3-1042620ed555-ovn-node-metrics-cert\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168652    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/67a066b6-b4af-49df-bc8b-5ae94c66ac84-cni-binary-copy\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168687    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cni-binary-copy\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168727    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-device-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.170875 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168735    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-sys\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168754    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168624    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-lib-modules\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168789    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-device-dir\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168780    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-cni-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168828    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgj6h\" (UniqueName: \"kubernetes.io/projected/67a066b6-b4af-49df-bc8b-5ae94c66ac84-kube-api-access-pgj6h\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168855    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e821c049-0ef1-44d3-8875-fae86b286c7b-tmp\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168880    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-kubelet\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168900    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/67a066b6-b4af-49df-bc8b-5ae94c66ac84-multus-cni-dir\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168907    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbwz\" (UniqueName: \"kubernetes.io/projected/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-kube-api-access-9hbwz\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168936    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3851429a-b5b9-4a29-b124-f84984721273-dbus\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168964    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fcfc6382-3050-4547-a388-ff49bbb472c4-konnectivity-ca\") pod \"konnectivity-agent-h7sbg\" (UID: \"fcfc6382-3050-4547-a388-ff49bbb472c4\") " pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168986    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-etc-selinux\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.168993    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-kubernetes\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169020    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-kubelet\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169053    2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169064    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-systemd\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169086    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-kubernetes\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.171558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169107    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3e0279f-9590-4e8e-bac3-1042620ed555-host-cni-netd\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.172280 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169020    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-systemd\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.172280 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169144    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3e0279f-9590-4e8e-bac3-1042620ed555-ovnkube-config\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.172280 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169158    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-cni-binary-copy\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.172280 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.169147    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93ced636-3c9e-428d-80e8-acb5ea71c7d2-host\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.172821 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.172792    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e821c049-0ef1-44d3-8875-fae86b286c7b-tmp\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.172927 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.172863    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3e0279f-9590-4e8e-bac3-1042620ed555-ovn-node-metrics-cert\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.173008 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.172942    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e821c049-0ef1-44d3-8875-fae86b286c7b-etc-tuned\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.177165 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.177144    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jtzv\" (UniqueName: \"kubernetes.io/projected/0d1c14b4-7aec-4c81-98c5-55841ddfa641-kube-api-access-2jtzv\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.177589 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.177565    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khw4k\" (UniqueName: \"kubernetes.io/projected/e2d0da7f-e17f-4522-bc6d-9d57c51f2d06-kube-api-access-khw4k\") pod \"node-resolver-dc22d\" (UID: \"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06\") " pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.178817 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.178793    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dxhc\" (UniqueName: \"kubernetes.io/projected/c3e0279f-9590-4e8e-bac3-1042620ed555-kube-api-access-2dxhc\") pod \"ovnkube-node-h6fb9\" (UID: \"c3e0279f-9590-4e8e-bac3-1042620ed555\") " pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.178925 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.178871    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5cps\" (UniqueName: \"kubernetes.io/projected/cd832b50-8ccc-4380-8ce0-4fe42ab5a939-kube-api-access-n5cps\") pod \"multus-additional-cni-plugins-w6zkl\" (UID: \"cd832b50-8ccc-4380-8ce0-4fe42ab5a939\") " pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.179647 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.179629    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgj6h\" (UniqueName: \"kubernetes.io/projected/67a066b6-b4af-49df-bc8b-5ae94c66ac84-kube-api-access-pgj6h\") pod \"multus-c7vfj\" (UID: \"67a066b6-b4af-49df-bc8b-5ae94c66ac84\") " pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.180022 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.180001    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptzp9\" (UniqueName: \"kubernetes.io/projected/4aef69b4-c167-4b19-98a6-f97b615b3d25-kube-api-access-ptzp9\") pod \"iptables-alerter-tzsln\" (UID: \"4aef69b4-c167-4b19-98a6-f97b615b3d25\") " pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.180342 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.180323    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzcrr\" (UniqueName: \"kubernetes.io/projected/e821c049-0ef1-44d3-8875-fae86b286c7b-kube-api-access-gzcrr\") pod \"tuned-8q2ls\" (UID: \"e821c049-0ef1-44d3-8875-fae86b286c7b\") " pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.180822 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.180802    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hbwz\" (UniqueName: \"kubernetes.io/projected/7b31f326-bed4-4f8a-8f7c-019ee5dea0e5-kube-api-access-9hbwz\") pod \"aws-ebs-csi-driver-node-g9zcz\" (UID: \"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.270372 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270337    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:39.270535 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270380    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fcfc6382-3050-4547-a388-ff49bbb472c4-agent-certs\") pod \"konnectivity-agent-h7sbg\" (UID: \"fcfc6382-3050-4547-a388-ff49bbb472c4\") " pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.270535 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270397    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93ced636-3c9e-428d-80e8-acb5ea71c7d2-serviceca\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.270535 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270432    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3851429a-b5b9-4a29-b124-f84984721273-dbus\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.270535 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270456    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fcfc6382-3050-4547-a388-ff49bbb472c4-konnectivity-ca\") pod \"konnectivity-agent-h7sbg\" (UID: \"fcfc6382-3050-4547-a388-ff49bbb472c4\") " pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.270535 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270482    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93ced636-3c9e-428d-80e8-acb5ea71c7d2-host\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.270535 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270524    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcpbn\" (UniqueName: \"kubernetes.io/projected/93ced636-3c9e-428d-80e8-acb5ea71c7d2-kube-api-access-zcpbn\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270550    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3851429a-b5b9-4a29-b124-f84984721273-dbus\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270571    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3851429a-b5b9-4a29-b124-f84984721273-kubelet-config\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270610    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93ced636-3c9e-428d-80e8-acb5ea71c7d2-host\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270618    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270631    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3851429a-b5b9-4a29-b124-f84984721273-kubelet-config\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.270740    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:39.270853 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.270813    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:39.77079311 +0000 UTC m=+3.074037867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:39.271136 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.270938    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/93ced636-3c9e-428d-80e8-acb5ea71c7d2-serviceca\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.271136 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.271073    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fcfc6382-3050-4547-a388-ff49bbb472c4-konnectivity-ca\") pod \"konnectivity-agent-h7sbg\" (UID: \"fcfc6382-3050-4547-a388-ff49bbb472c4\") " pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.273218 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.273185    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fcfc6382-3050-4547-a388-ff49bbb472c4-agent-certs\") pod \"konnectivity-agent-h7sbg\" (UID: \"fcfc6382-3050-4547-a388-ff49bbb472c4\") " pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.279534 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.279513    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:57:39.279534 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.279538    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:57:39.279729 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.279551    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:39.279729 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.279614    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:57:39.779598103 +0000 UTC m=+3.082842866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:39.283094 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.283073    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcpbn\" (UniqueName: \"kubernetes.io/projected/93ced636-3c9e-428d-80e8-acb5ea71c7d2-kube-api-access-zcpbn\") pod \"node-ca-qnt7r\" (UID: \"93ced636-3c9e-428d-80e8-acb5ea71c7d2\") " pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.353441 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.353379    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8q2ls"
Jul 27 20:57:39.361490 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.361471    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w6zkl"
Jul 27 20:57:39.373617 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.373601    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dc22d"
Jul 27 20:57:39.378414 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.378391    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz"
Jul 27 20:57:39.384484 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.384464    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c7vfj"
Jul 27 20:57:39.394220 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.394201    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tzsln"
Jul 27 20:57:39.400466 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.400449    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:57:39.407409 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.407389    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:57:39.410395 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.410378    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qnt7r"
Jul 27 20:57:39.674320 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.674226    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:39.674475 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.674383    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:39.674475 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.674468    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:40.674436454 +0000 UTC m=+3.977681227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:39.774766 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.774734    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:39.774915 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.774902    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:39.774986 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.774973    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:40.774953586 +0000 UTC m=+4.078198334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:39.875245 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:39.875197    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:39.875423 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.875372    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:57:39.875423 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.875400    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:57:39.875423 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.875414    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:39.875568 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:39.875480    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:57:40.875460861 +0000 UTC m=+4.178705614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:40.095853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.095814    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 20:52:39 +0000 UTC" deadline="2028-01-08 11:06:16.05123144 +0000 UTC"
Jul 27 20:57:40.095853 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.095849    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12710h8m35.955387228s"
Jul 27 20:57:40.178812 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.178784    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d0da7f_e17f_4522_bc6d_9d57c51f2d06.slice/crio-2abbe2a5a94f3c40f31d761686a4a07da14d5eb473483f9a2bccdabdaa6e103e WatchSource:0}: Error finding container 2abbe2a5a94f3c40f31d761686a4a07da14d5eb473483f9a2bccdabdaa6e103e: Status 404 returned error can't find the container with id 2abbe2a5a94f3c40f31d761686a4a07da14d5eb473483f9a2bccdabdaa6e103e
Jul 27 20:57:40.180126 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.180081    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e0279f_9590_4e8e_bac3_1042620ed555.slice/crio-55e7396c974542f99a5b5c4dde617f51e2e8485a8eccbef664763bbd7f4df54d WatchSource:0}: Error finding container 55e7396c974542f99a5b5c4dde617f51e2e8485a8eccbef664763bbd7f4df54d: Status 404 returned error can't find the container with id 55e7396c974542f99a5b5c4dde617f51e2e8485a8eccbef664763bbd7f4df54d
Jul 27 20:57:40.183966 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.183941    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a066b6_b4af_49df_bc8b_5ae94c66ac84.slice/crio-2ddb6e388a8c143a85fde8b8a36f6c21e134b838a41d44353a7f70124d987060 WatchSource:0}: Error finding container 2ddb6e388a8c143a85fde8b8a36f6c21e134b838a41d44353a7f70124d987060: Status 404 returned error can't find the container with id 2ddb6e388a8c143a85fde8b8a36f6c21e134b838a41d44353a7f70124d987060
Jul 27 20:57:40.185741 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.185719    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ced636_3c9e_428d_80e8_acb5ea71c7d2.slice/crio-c24e1e71c00c9e35a60b210eeb7bea64b82f3ef67e8682f790e2cd3022f32976 WatchSource:0}: Error finding container c24e1e71c00c9e35a60b210eeb7bea64b82f3ef67e8682f790e2cd3022f32976: Status 404 returned error can't find the container with id c24e1e71c00c9e35a60b210eeb7bea64b82f3ef67e8682f790e2cd3022f32976
Jul 27 20:57:40.186535 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.186511    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode821c049_0ef1_44d3_8875_fae86b286c7b.slice/crio-2ea716bbfb03d3dad2a200faff30e57ae4fd98012499a7bbd957cab171d87142 WatchSource:0}: Error finding container 2ea716bbfb03d3dad2a200faff30e57ae4fd98012499a7bbd957cab171d87142: Status 404 returned error can't find the container with id 2ea716bbfb03d3dad2a200faff30e57ae4fd98012499a7bbd957cab171d87142
Jul 27 20:57:40.188292 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.188274    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b31f326_bed4_4f8a_8f7c_019ee5dea0e5.slice/crio-7ab8e0c055aa531e31b6d2dd451a147a9a374aff8d9ef70944676accb645448a WatchSource:0}: Error finding container 7ab8e0c055aa531e31b6d2dd451a147a9a374aff8d9ef70944676accb645448a: Status 404 returned error can't find the container with id 7ab8e0c055aa531e31b6d2dd451a147a9a374aff8d9ef70944676accb645448a
Jul 27 20:57:40.191773 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.191750    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:40.191917 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.191889    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:40.193832 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.193795    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd832b50_8ccc_4380_8ce0_4fe42ab5a939.slice/crio-fd28071621c913b5a201136a28d129a7eacce92d1759435a1f75b3ff40f31a15 WatchSource:0}: Error finding container fd28071621c913b5a201136a28d129a7eacce92d1759435a1f75b3ff40f31a15: Status 404 returned error can't find the container with id fd28071621c913b5a201136a28d129a7eacce92d1759435a1f75b3ff40f31a15
Jul 27 20:57:40.194616 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.194561    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aef69b4_c167_4b19_98a6_f97b615b3d25.slice/crio-7f01d107ef99e84a088c305757fea9b7884d05918e0c117c84d5d2f661c46103 WatchSource:0}: Error finding container 7f01d107ef99e84a088c305757fea9b7884d05918e0c117c84d5d2f661c46103: Status 404 returned error can't find the container with id 7f01d107ef99e84a088c305757fea9b7884d05918e0c117c84d5d2f661c46103
Jul 27 20:57:40.197539 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:57:40.197515    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfc6382_3050_4547_a388_ff49bbb472c4.slice/crio-146b36fb509d28f04cc49a740108e96b1ea0b6748e81871f6ffba883e279d2e7 WatchSource:0}: Error finding container 146b36fb509d28f04cc49a740108e96b1ea0b6748e81871f6ffba883e279d2e7: Status 404 returned error can't find the container with id 146b36fb509d28f04cc49a740108e96b1ea0b6748e81871f6ffba883e279d2e7
Jul 27 20:57:40.199917 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.199889    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerStarted","Data":"fd28071621c913b5a201136a28d129a7eacce92d1759435a1f75b3ff40f31a15"}
Jul 27 20:57:40.201384 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.201349    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7vfj" event={"ID":"67a066b6-b4af-49df-bc8b-5ae94c66ac84","Type":"ContainerStarted","Data":"2ddb6e388a8c143a85fde8b8a36f6c21e134b838a41d44353a7f70124d987060"}
Jul 27 20:57:40.203353 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.203216    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"55e7396c974542f99a5b5c4dde617f51e2e8485a8eccbef664763bbd7f4df54d"}
Jul 27 20:57:40.204683 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.204633    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tzsln" event={"ID":"4aef69b4-c167-4b19-98a6-f97b615b3d25","Type":"ContainerStarted","Data":"7f01d107ef99e84a088c305757fea9b7884d05918e0c117c84d5d2f661c46103"}
Jul 27 20:57:40.205997 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.205977    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz" event={"ID":"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5","Type":"ContainerStarted","Data":"7ab8e0c055aa531e31b6d2dd451a147a9a374aff8d9ef70944676accb645448a"}
Jul 27 20:57:40.207278 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.207240    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8q2ls" event={"ID":"e821c049-0ef1-44d3-8875-fae86b286c7b","Type":"ContainerStarted","Data":"2ea716bbfb03d3dad2a200faff30e57ae4fd98012499a7bbd957cab171d87142"}
Jul 27 20:57:40.208551 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.208530    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qnt7r" event={"ID":"93ced636-3c9e-428d-80e8-acb5ea71c7d2","Type":"ContainerStarted","Data":"c24e1e71c00c9e35a60b210eeb7bea64b82f3ef67e8682f790e2cd3022f32976"}
Jul 27 20:57:40.209773 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.209750    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dc22d" event={"ID":"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06","Type":"ContainerStarted","Data":"2abbe2a5a94f3c40f31d761686a4a07da14d5eb473483f9a2bccdabdaa6e103e"}
Jul 27 20:57:40.684169 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.684082    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:40.684358 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.684252    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:40.684358 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.684340    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:42.68432011 +0000 UTC m=+5.987564857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:40.784720 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.784640    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:40.784902 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.784825    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:40.784902 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.784888    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:42.784869423 +0000 UTC m=+6.088114175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:40.885551 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:40.885507    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:40.885741 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.885716    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:57:40.885741 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.885739    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:57:40.885862 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.885752    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:40.885862 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:40.885814    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:57:42.885795667 +0000 UTC m=+6.189040418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:41.097134 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.096943    2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 20:52:39 +0000 UTC" deadline="2028-03-15 00:03:54.224472901 +0000 UTC"
Jul 27 20:57:41.097134 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.097016    2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14307h6m13.127461595s"
Jul 27 20:57:41.194547 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.194491    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:41.194724 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:41.194622    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:41.195076 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.195050    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:41.195186 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:41.195163    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:41.226977 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.226940    2552 generic.go:356] "Generic (PLEG): container finished" podID="ce115a00c211e7ec27fc8ba3d423f461" containerID="f402f5f5d6fed1825ae1f4f259aad8a732977f75d4e153768600bb1403caaeb0" exitCode=0
Jul 27 20:57:41.227361 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.227326    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal" event={"ID":"ce115a00c211e7ec27fc8ba3d423f461","Type":"ContainerDied","Data":"f402f5f5d6fed1825ae1f4f259aad8a732977f75d4e153768600bb1403caaeb0"}
Jul 27 20:57:41.264709 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.264681    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal" event={"ID":"673a856a2cbcee0e523f1fb250518aab","Type":"ContainerStarted","Data":"2ca4062f1a6eeba876dcf22465fba524fd2830bcf2ad3ae0cb46bb5982290a92"}
Jul 27 20:57:41.272596 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:41.272547    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h7sbg" event={"ID":"fcfc6382-3050-4547-a388-ff49bbb472c4","Type":"ContainerStarted","Data":"146b36fb509d28f04cc49a740108e96b1ea0b6748e81871f6ffba883e279d2e7"}
Jul 27 20:57:42.192498 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.191942    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:42.192498 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.192087    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:42.300102 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.300056    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal" event={"ID":"ce115a00c211e7ec27fc8ba3d423f461","Type":"ContainerStarted","Data":"5a75ccd6f66dfe54a2cc872425830fe19b2085ee540e2d8c68a52e6fd927b52b"}
Jul 27 20:57:42.317131 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.316415    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-33.ec2.internal" podStartSLOduration=4.316394064 podStartE2EDuration="4.316394064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:57:42.315734995 +0000 UTC m=+5.618979766" watchObservedRunningTime="2026-07-27 20:57:42.316394064 +0000 UTC m=+5.619638833"
Jul 27 20:57:42.317131 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.316918    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-33.ec2.internal" podStartSLOduration=4.316906509 podStartE2EDuration="4.316906509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:57:41.280411524 +0000 UTC m=+4.583656294" watchObservedRunningTime="2026-07-27 20:57:42.316906509 +0000 UTC m=+5.620151278"
Jul 27 20:57:42.701436 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.701400    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:42.701616 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.701582    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:42.701697 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.701649    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:46.701629483 +0000 UTC m=+10.004874229 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:42.802118 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.802076    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:42.802343 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.802277    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:42.802411 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.802380    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:46.802360422 +0000 UTC m=+10.105605170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:42.903522 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:42.902877    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:42.903522 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.903056    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:57:42.903522 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.903076    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:57:42.903522 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.903090    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:42.903522 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:42.903148    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:57:46.903128926 +0000 UTC m=+10.206373673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:43.194497 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:43.194371    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:43.194954 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:43.194513    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:43.194954 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:43.194835    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:43.194954 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:43.194938    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:44.191749 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:44.191715    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:44.191951 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:44.191867    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:45.192467 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:45.192433    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:45.192952 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:45.192582    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:45.193138 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:45.193119    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:45.193265 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:45.193225    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:46.192144 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:46.192103    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:46.192360 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.192280    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:46.736897 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:46.736863    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:46.737325 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.736990    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:46.737325 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.737052    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:54.737033916 +0000 UTC m=+18.040278663 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:46.837419 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:46.837383    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:46.837606 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.837540    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:46.837606 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.837596    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:57:54.837577898 +0000 UTC m=+18.140822641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:46.938626 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:46.938591    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:46.938834 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.938769    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:57:46.938834 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.938792    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:57:46.938834 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.938805    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:46.938997 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:46.938859    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:57:54.938843038 +0000 UTC m=+18.242087795 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:47.192963 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:47.192895    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:47.193081 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:47.192991    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:47.193468 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:47.193450    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:47.193572 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:47.193552    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:48.191912 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:48.191876    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:48.192314 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:48.192017    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:49.191784 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:49.191708    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:49.191924 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:49.191832    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:49.192162 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:49.191714    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:49.192529 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:49.192264    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:50.191394 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:50.191348    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:50.191571 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:50.191505    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:51.192143 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:51.192112    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:51.192608 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:51.192254    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:51.192608 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:51.192296    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:51.192608 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:51.192411    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:52.191585 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:52.191553    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:52.191744 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:52.191714    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:53.192014 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:53.191981    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:53.192516 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:53.192110    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:53.192516 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:53.192173    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:53.192516 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:53.192370    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:54.192131 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:54.192098    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:54.192626 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:54.192237    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:54.804438 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:54.804407    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:54.804604 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:54.804536    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:54.804604 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:54.804597    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:10.804582032 +0000 UTC m=+34.107826792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:57:54.904890 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:54.904859    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:54.905076 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:54.904992    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:54.905076 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:54.905051    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:10.905035286 +0000 UTC m=+34.208280028 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:57:55.005680 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:55.005644    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:55.005872 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:55.005841    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:57:55.005872 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:55.005866    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:57:55.005958 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:55.005879    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:55.005958 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:55.005938    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:58:11.005918475 +0000 UTC m=+34.309163220 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:57:55.191960 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:55.191928    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:55.191960 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:55.191947    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:55.192104 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:55.192056    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:55.192171 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:55.192155    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:56.191432 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:56.191398    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:56.191634 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:56.191521    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:57.194787 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:57.194750    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:57.195127 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:57.194895    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:57.195395 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:57.195376    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:57.195587 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:57.195569    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:58.191769 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:58.191736    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:57:58.191954 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:58.191886    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:57:59.191627 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:59.191585    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:57:59.191627 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:59.191605    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:57:59.192368 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:59.191772    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:57:59.192368 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:57:59.191870    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:57:59.338319 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:59.337703    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h7sbg" event={"ID":"fcfc6382-3050-4547-a388-ff49bbb472c4","Type":"ContainerStarted","Data":"9c5988f24f628c73087e4a733e32032850c9d731b3f47a1d064298a9bade0bd1"}
Jul 27 20:57:59.353704 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:57:59.353010    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h7sbg" podStartSLOduration=6.397921416 podStartE2EDuration="21.352992402s" totalImagesPullingTime="14.955070986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:57:59.352579476 +0000 UTC m=+22.655824242" watchObservedRunningTime="2026-07-27 20:57:59.352992402 +0000 UTC m=+22.656237169"
Jul 27 20:58:00.192118 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.191905    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:00.192792 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:00.192162    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:00.341929 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.341887    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz" event={"ID":"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5","Type":"ContainerStarted","Data":"bbb5a7a7fbe2cb02f8f317bf8c21eb6fea8b96fdf30c87b1a7acbc7bff3fa49b"}
Jul 27 20:58:00.343636 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.343585    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8q2ls" event={"ID":"e821c049-0ef1-44d3-8875-fae86b286c7b","Type":"ContainerStarted","Data":"1fd96ca1c6eea58f1aef6a6668ff0b31e3ce7073f581d69291eff05966cad6f0"}
Jul 27 20:58:00.345502 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.345474    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qnt7r" event={"ID":"93ced636-3c9e-428d-80e8-acb5ea71c7d2","Type":"ContainerStarted","Data":"8839c43ca0e4cab7a3ebb7386c08c169a5a8ff2ecb4a87d7b9e5e9de2411efdb"}
Jul 27 20:58:00.347136 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.347088    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dc22d" event={"ID":"e2d0da7f-e17f-4522-bc6d-9d57c51f2d06","Type":"ContainerStarted","Data":"ed9cd9fc2f7b870ed6e47555899eb7af0774ad883a3a9c7217c32af6bc545b70"}
Jul 27 20:58:00.348802 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.348775    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd832b50-8ccc-4380-8ce0-4fe42ab5a939" containerID="f468d87923417ddc194064d08da685cc4f5fe18b5d4873483db26e96070d5c62" exitCode=0
Jul 27 20:58:00.348910 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.348861    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerDied","Data":"f468d87923417ddc194064d08da685cc4f5fe18b5d4873483db26e96070d5c62"}
Jul 27 20:58:00.351203 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.351181    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c7vfj" event={"ID":"67a066b6-b4af-49df-bc8b-5ae94c66ac84","Type":"ContainerStarted","Data":"56294f3cd039afa46a13ee7ddfbac1d7a317788a02d690c395f2b3008824e55e"}
Jul 27 20:58:00.354805 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.354785    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"b6d09d35fef46f67fa4fa1608727078fb32bed3e909e79371329ef2652c499b1"}
Jul 27 20:58:00.354904 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.354849    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"99202b80d8d3ef645cb183d86d3f972af3831fd9fad22e25a062739beb11824e"}
Jul 27 20:58:00.354904 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.354889    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"239738df13bd212448a07148b2c84253bc2237951c2cd6e9052a6d8cb224159c"}
Jul 27 20:58:00.355005 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.354934    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"b1ac1dbdb917e09103306ba483e2573f6d7ee3267c7a1a11037a1ccb932a4ef9"}
Jul 27 20:58:00.355005 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.354975    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"b2733409d98d11dd04a79e3e40c2db727930e5d697b6c2caeede6573820f91d0"}
Jul 27 20:58:00.355097 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.355018    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"b8cb660911346d776c6470cc2ede08349767b9a1a6be9de16bebafc101dcbbd9"}
Jul 27 20:58:00.360703 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.360664    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8q2ls" podStartSLOduration=4.31696055 podStartE2EDuration="23.360651624s" totalImagesPullingTime="19.043691074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:00.360390913 +0000 UTC m=+23.663635690" watchObservedRunningTime="2026-07-27 20:58:00.360651624 +0000 UTC m=+23.663896389"
Jul 27 20:58:00.378619 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.378578    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-c7vfj" podStartSLOduration=4.301724352 podStartE2EDuration="23.378566014s" totalImagesPullingTime="19.076841662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:00.378543459 +0000 UTC m=+23.681788226" watchObservedRunningTime="2026-07-27 20:58:00.378566014 +0000 UTC m=+23.681810779"
Jul 27 20:58:00.392891 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.392859    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dc22d" podStartSLOduration=4.363044493 podStartE2EDuration="23.392847156s" totalImagesPullingTime="19.029802663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:00.392423418 +0000 UTC m=+23.695668183" watchObservedRunningTime="2026-07-27 20:58:00.392847156 +0000 UTC m=+23.696091921"
Jul 27 20:58:00.424927 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.424890    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qnt7r" podStartSLOduration=3.407931851 podStartE2EDuration="22.424877733s" totalImagesPullingTime="19.016945882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:00.424568366 +0000 UTC m=+23.727813130" watchObservedRunningTime="2026-07-27 20:58:00.424877733 +0000 UTC m=+23.728122502"
Jul 27 20:58:00.862003 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:00.861975    2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 27 20:58:01.128908 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.128745    2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T20:58:00.861995465Z","UUID":"fce9256f-b571-4ac2-b866-f997688abdee","Handler":null,"Name":"","Endpoint":""}
Jul 27 20:58:01.130354 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.130328    2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 27 20:58:01.130503 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.130381    2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 27 20:58:01.191999 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.191971    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:01.192170 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.192005    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:01.192170 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:01.192095    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:58:01.192571 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:01.192264    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:58:01.196090 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.196042    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:58:01.196584 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.196567    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:58:01.360008 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.359981    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tzsln" event={"ID":"4aef69b4-c167-4b19-98a6-f97b615b3d25","Type":"ContainerStarted","Data":"c4574cd0d27a66694e1dd9de7b821131000aa1fdb0fdbef4b81ad24484d58e72"}
Jul 27 20:58:01.362446 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.362420    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz" event={"ID":"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5","Type":"ContainerStarted","Data":"2e16449d105e4f58e6a0be0d8b67bd8025f09de7a410583c5aed834aac7d1615"}
Jul 27 20:58:01.364132 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.364111    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:58:01.364512 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.364497    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h7sbg"
Jul 27 20:58:01.376738 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:01.376692    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tzsln" podStartSLOduration=5.367914062 podStartE2EDuration="24.376675776s" totalImagesPullingTime="19.008761714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:01.376120522 +0000 UTC m=+24.679365288" watchObservedRunningTime="2026-07-27 20:58:01.376675776 +0000 UTC m=+24.679920542"
Jul 27 20:58:02.191962 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:02.191903    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:02.192138 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:02.192015    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:02.366534 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:02.366498    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz" event={"ID":"7b31f326-bed4-4f8a-8f7c-019ee5dea0e5","Type":"ContainerStarted","Data":"f06fb6b3006b6e178900632d273093a9efe2187cd4aad44eba2db215e89e12b3"}
Jul 27 20:58:02.370388 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:02.370347    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"7d0d61ea092605e95c24dde970d8463618ffd05cae81972ecc35a0bc37e6b866"}
Jul 27 20:58:02.383878 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:02.383829    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g9zcz" podStartSLOduration=3.976602784 podStartE2EDuration="25.383809977s" totalImagesPullingTime="21.407207193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:02.382418551 +0000 UTC m=+25.685663318" watchObservedRunningTime="2026-07-27 20:58:02.383809977 +0000 UTC m=+25.687054745"
Jul 27 20:58:03.191920 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:03.191705    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:03.192097 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:03.191782    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:03.192097 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:03.192017    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:58:03.192207 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:03.192106    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:58:04.191971 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:04.191943    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:04.192332 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:04.192056    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:05.194938 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.194871    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:05.195457 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:05.194979    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:58:05.195457 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.195026    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:05.195457 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:05.195105    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:58:05.378682 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.378480    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd832b50-8ccc-4380-8ce0-4fe42ab5a939" containerID="78269e2df4a478af403afcfe0a9756de8723b5a3898fe9a3826b3817bd5aab8f" exitCode=0
Jul 27 20:58:05.378862 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.378568    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerDied","Data":"78269e2df4a478af403afcfe0a9756de8723b5a3898fe9a3826b3817bd5aab8f"}
Jul 27 20:58:05.382534 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.382514    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" event={"ID":"c3e0279f-9590-4e8e-bac3-1042620ed555","Type":"ContainerStarted","Data":"130ca31664508557b2d4d544f24cbc37174b6f975af8611659e5eb05a4f29462"}
Jul 27 20:58:05.382804 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.382788    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:58:05.398995 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.398976    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:58:05.451212 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:05.451116    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9" podStartSLOduration=9.348528675 podStartE2EDuration="28.4511034s" totalImagesPullingTime="19.102574725s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:05.426806818 +0000 UTC m=+28.730051576" watchObservedRunningTime="2026-07-27 20:58:05.4511034 +0000 UTC m=+28.754348164"
Jul 27 20:58:06.192066 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.192036    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:06.192345 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:06.192171    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:06.384979 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.384960    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:58:06.385218 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.385047    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:58:06.400395 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.400375    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:58:06.778349 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.778271    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dp2gb"]
Jul 27 20:58:06.778481 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.778397    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:06.778518 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:06.778476    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:58:06.781430 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.781406    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cgq4s"]
Jul 27 20:58:06.781574 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.781493    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:06.781648 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:06.781572    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:06.782076 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.782055    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4x6mt"]
Jul 27 20:58:06.782207 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:06.782170    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:06.782282 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:06.782260    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:58:07.388981 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:07.388855    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd832b50-8ccc-4380-8ce0-4fe42ab5a939" containerID="92ac72af9f8457a00b7db9779e21f7e99b21de28acf7b121a5fbc309a2fcdf22" exitCode=0
Jul 27 20:58:07.388981 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:07.388912    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerDied","Data":"92ac72af9f8457a00b7db9779e21f7e99b21de28acf7b121a5fbc309a2fcdf22"}
Jul 27 20:58:08.191581 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:08.191520    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:08.191581 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:08.191546    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:08.191815 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:08.191658    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:08.191815 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:08.191785    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:58:09.192262 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:09.192173    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:09.192757 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:09.192316    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:58:09.395178 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:09.395130    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd832b50-8ccc-4380-8ce0-4fe42ab5a939" containerID="dcb88c54a38bb2910a8f2d6ed64c9dbf7919a670f739c35a863678feb0805988" exitCode=0
Jul 27 20:58:09.395178 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:09.395171    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerDied","Data":"dcb88c54a38bb2910a8f2d6ed64c9dbf7919a670f739c35a863678feb0805988"}
Jul 27 20:58:10.191776 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:10.191744    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:10.191947 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:10.191753    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:10.191947 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:10.191865    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4x6mt" podUID="2ee93eb9-66cf-4adb-b494-72fd0d8d220c"
Jul 27 20:58:10.192064 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:10.191978    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 20:58:10.823241 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:10.823189    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:10.823896 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:10.823344    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:58:10.823896 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:10.823410    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:42.823391982 +0000 UTC m=+66.126636725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 27 20:58:10.924507 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:10.924457    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:10.924680 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:10.924596    2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 27 20:58:10.924680 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:10.924652    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret podName:3851429a-b5b9-4a29-b124-f84984721273 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:42.924637143 +0000 UTC m=+66.227881886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret") pod "global-pull-secret-syncer-dp2gb" (UID: "3851429a-b5b9-4a29-b124-f84984721273") : object "kube-system"/"original-pull-secret" not registered
Jul 27 20:58:11.024773 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:11.024745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:11.024947 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:11.024924    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 27 20:58:11.024947 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:11.024947    2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 27 20:58:11.025062 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:11.024959    2552 projected.go:196] Error preparing data for projected volume kube-api-access-lrckv for pod openshift-network-diagnostics/network-check-target-4x6mt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:58:11.025062 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:11.025020    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv podName:2ee93eb9-66cf-4adb-b494-72fd0d8d220c nodeName:}" failed. No retries permitted until 2026-07-27 20:58:43.025001672 +0000 UTC m=+66.328246429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lrckv" (UniqueName: "kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv") pod "network-check-target-4x6mt" (UID: "2ee93eb9-66cf-4adb-b494-72fd0d8d220c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 27 20:58:11.191752 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:11.191672    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:11.191914 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:11.191788    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dp2gb" podUID="3851429a-b5b9-4a29-b124-f84984721273"
Jul 27 20:58:12.020431 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.020351    2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-33.ec2.internal" event="NodeReady"
Jul 27 20:58:12.020804 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.020497    2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 27 20:58:12.060264 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.060215    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mfplf"]
Jul 27 20:58:12.063933 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.063909    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bkp2g"]
Jul 27 20:58:12.064077 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.064056    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.066136 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.066117    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.066911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.066888    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 27 20:58:12.066911 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.066877    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 27 20:58:12.067088 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.066998    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bvklp\""
Jul 27 20:58:12.068500 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.068483    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5mn65\""
Jul 27 20:58:12.068765 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.068750    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 27 20:58:12.068765 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.068762    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 27 20:58:12.068892 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.068787    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 27 20:58:12.073404 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.073376    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mfplf"]
Jul 27 20:58:12.088978 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.088957    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bkp2g"]
Jul 27 20:58:12.191856 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.191831    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:12.192004 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.191988    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:12.195374 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.195349    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cxsx8\""
Jul 27 20:58:12.195513 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.195397    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 20:58:12.195564 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.195549    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hn7dl\""
Jul 27 20:58:12.195606 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.195599    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 20:58:12.195744 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.195728    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 20:58:12.232678 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.232654    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b0321d-1d87-409a-9277-a7002c2f80b8-tmp-dir\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.232799 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.232697    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpv9d\" (UniqueName: \"kubernetes.io/projected/a5b0321d-1d87-409a-9277-a7002c2f80b8-kube-api-access-gpv9d\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.232799 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.232725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b0321d-1d87-409a-9277-a7002c2f80b8-config-volume\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.232799 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.232788    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7q74\" (UniqueName: \"kubernetes.io/projected/7c8704a1-3d26-44a4-bb4c-0659e2702945-kube-api-access-p7q74\") pod \"ingress-canary-bkp2g\" (UID: \"7c8704a1-3d26-44a4-bb4c-0659e2702945\") " pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.232955 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.232813    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.232955 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.232834    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c8704a1-3d26-44a4-bb4c-0659e2702945-cert\") pod \"ingress-canary-bkp2g\" (UID: \"7c8704a1-3d26-44a4-bb4c-0659e2702945\") " pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.333962 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.333891    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7q74\" (UniqueName: \"kubernetes.io/projected/7c8704a1-3d26-44a4-bb4c-0659e2702945-kube-api-access-p7q74\") pod \"ingress-canary-bkp2g\" (UID: \"7c8704a1-3d26-44a4-bb4c-0659e2702945\") " pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.333962 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.333923    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.333962 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.333951    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c8704a1-3d26-44a4-bb4c-0659e2702945-cert\") pod \"ingress-canary-bkp2g\" (UID: \"7c8704a1-3d26-44a4-bb4c-0659e2702945\") " pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.334247 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.334045    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b0321d-1d87-409a-9277-a7002c2f80b8-tmp-dir\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.334247 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:12.334064    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:12.334247 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.334077    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpv9d\" (UniqueName: \"kubernetes.io/projected/a5b0321d-1d87-409a-9277-a7002c2f80b8-kube-api-access-gpv9d\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.334247 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.334101    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b0321d-1d87-409a-9277-a7002c2f80b8-config-volume\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.334247 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:12.334140    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:12.834120355 +0000 UTC m=+36.137365097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:12.334576 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.334418    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5b0321d-1d87-409a-9277-a7002c2f80b8-tmp-dir\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.334654 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.334633    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a5b0321d-1d87-409a-9277-a7002c2f80b8-config-volume\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.338659 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.338635    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c8704a1-3d26-44a4-bb4c-0659e2702945-cert\") pod \"ingress-canary-bkp2g\" (UID: \"7c8704a1-3d26-44a4-bb4c-0659e2702945\") " pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.344604 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.344579    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpv9d\" (UniqueName: \"kubernetes.io/projected/a5b0321d-1d87-409a-9277-a7002c2f80b8-kube-api-access-gpv9d\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.344707 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.344590    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7q74\" (UniqueName: \"kubernetes.io/projected/7c8704a1-3d26-44a4-bb4c-0659e2702945-kube-api-access-p7q74\") pod \"ingress-canary-bkp2g\" (UID: \"7c8704a1-3d26-44a4-bb4c-0659e2702945\") " pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.384628 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.384602    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bkp2g"
Jul 27 20:58:12.839216 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:12.839181    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:12.839373 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:12.839358    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:12.839427 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:12.839421    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:13.839404315 +0000 UTC m=+37.142649061 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:13.191779 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:13.191707    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:13.194888 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:13.194851    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 20:58:13.331599 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:13.331338    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bkp2g"]
Jul 27 20:58:13.334342 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:58:13.334318    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c8704a1_3d26_44a4_bb4c_0659e2702945.slice/crio-210a55f9322dcde4de0a6fbd38e18718bf3cd13a394306480d5b15a1aad3e01b WatchSource:0}: Error finding container 210a55f9322dcde4de0a6fbd38e18718bf3cd13a394306480d5b15a1aad3e01b: Status 404 returned error can't find the container with id 210a55f9322dcde4de0a6fbd38e18718bf3cd13a394306480d5b15a1aad3e01b
Jul 27 20:58:13.406559 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:13.406528    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bkp2g" event={"ID":"7c8704a1-3d26-44a4-bb4c-0659e2702945","Type":"ContainerStarted","Data":"210a55f9322dcde4de0a6fbd38e18718bf3cd13a394306480d5b15a1aad3e01b"}
Jul 27 20:58:13.409193 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:13.409168    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerStarted","Data":"739305f3a7da995bf7743892a4d4727f40ad683c895c7974211ead3a73f4a79c"}
Jul 27 20:58:13.846096 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:13.846008    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:13.846287 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:13.846183    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:13.846359 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:13.846299    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:15.846277703 +0000 UTC m=+39.149522445 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:14.414428 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:14.414378    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd832b50-8ccc-4380-8ce0-4fe42ab5a939" containerID="739305f3a7da995bf7743892a4d4727f40ad683c895c7974211ead3a73f4a79c" exitCode=0
Jul 27 20:58:14.414847 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:14.414438    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerDied","Data":"739305f3a7da995bf7743892a4d4727f40ad683c895c7974211ead3a73f4a79c"}
Jul 27 20:58:15.420051 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:15.420020    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd832b50-8ccc-4380-8ce0-4fe42ab5a939" containerID="dea976fb27e79573f1794a01fdb59a31838d1b35c56c8e16763acdc965d3fc31" exitCode=0
Jul 27 20:58:15.420451 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:15.420061    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerDied","Data":"dea976fb27e79573f1794a01fdb59a31838d1b35c56c8e16763acdc965d3fc31"}
Jul 27 20:58:15.862469 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:15.862170    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:15.862691 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:15.862673    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:15.862751 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:15.862734    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:19.862717519 +0000 UTC m=+43.165962268 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:16.423340 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:16.423306    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bkp2g" event={"ID":"7c8704a1-3d26-44a4-bb4c-0659e2702945","Type":"ContainerStarted","Data":"b903ba42789783380d4f5ae4298eb3ec6051b1a82a99e3579c6ad81f304b8c9c"}
Jul 27 20:58:16.426558 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:16.426535    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" event={"ID":"cd832b50-8ccc-4380-8ce0-4fe42ab5a939","Type":"ContainerStarted","Data":"30e535b73f94751f150035931b672a92c837c6d58518441e967d2a02ffb8ef21"}
Jul 27 20:58:16.439166 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:16.439130    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bkp2g" podStartSLOduration=2.184778017 podStartE2EDuration="4.439112914s" totalImagesPullingTime="2.254334897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:16.439083445 +0000 UTC m=+39.742328221" watchObservedRunningTime="2026-07-27 20:58:16.439112914 +0000 UTC m=+39.742357677"
Jul 27 20:58:16.459839 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:16.459791    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w6zkl" podStartSLOduration=10.143138321 podStartE2EDuration="39.459776728s" totalImagesPullingTime="29.316638407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:16.458249602 +0000 UTC m=+39.761494367" watchObservedRunningTime="2026-07-27 20:58:16.459776728 +0000 UTC m=+39.763021493"
Jul 27 20:58:19.890209 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:19.890164    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:19.890710 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:19.890314    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:19.890710 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:19.890397    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:27.89037433 +0000 UTC m=+51.193619079 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:27.948923 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:27.948886    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:27.949411 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:27.949030    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:27.949411 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:27.949091    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:58:43.949075088 +0000 UTC m=+67.252319831 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:36.674293 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.674257    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"]
Jul 27 20:58:36.676621 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.676603    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.679200 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.679180    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 27 20:58:36.680293 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.680274    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 27 20:58:36.680367 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.680307    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 27 20:58:36.680367 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.680348    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 27 20:58:36.680576 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.680561    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 27 20:58:36.680661 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.680644    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 27 20:58:36.680711 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.680644    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 27 20:58:36.691655 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.691633    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"]
Jul 27 20:58:36.706327 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.706298    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.706455 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.706341    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.706455 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.706389    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-hub\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.706455 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.706412    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/758a99e3-bc0c-48b1-890b-fd6f037e37d3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.706455 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.706444    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-ca\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.706601 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.706467    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gstz\" (UniqueName: \"kubernetes.io/projected/758a99e3-bc0c-48b1-890b-fd6f037e37d3-kube-api-access-6gstz\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.807272 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807236    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.807435 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807290    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.807435 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807330    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-hub\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.807435 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807351    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/758a99e3-bc0c-48b1-890b-fd6f037e37d3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.807435 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807382    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-ca\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.807435 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807405    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gstz\" (UniqueName: \"kubernetes.io/projected/758a99e3-bc0c-48b1-890b-fd6f037e37d3-kube-api-access-6gstz\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.808008 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.807986    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/758a99e3-bc0c-48b1-890b-fd6f037e37d3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.811451 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.811430    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-hub\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.811451 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.811442    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-ca\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.811571 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.811464    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.811571 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.811558    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/758a99e3-bc0c-48b1-890b-fd6f037e37d3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.818463 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.818442    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gstz\" (UniqueName: \"kubernetes.io/projected/758a99e3-bc0c-48b1-890b-fd6f037e37d3-kube-api-access-6gstz\") pod \"cluster-proxy-proxy-agent-76478669c5-dcxb5\" (UID: \"758a99e3-bc0c-48b1-890b-fd6f037e37d3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:36.998815 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:36.998737    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 20:58:37.130490 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:37.130461    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"]
Jul 27 20:58:37.133723 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:58:37.133698    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod758a99e3_bc0c_48b1_890b_fd6f037e37d3.slice/crio-749d4f3f361ac172d0d86ddff209b4cb25df1182b8112965c86ad0f986ba4469 WatchSource:0}: Error finding container 749d4f3f361ac172d0d86ddff209b4cb25df1182b8112965c86ad0f986ba4469: Status 404 returned error can't find the container with id 749d4f3f361ac172d0d86ddff209b4cb25df1182b8112965c86ad0f986ba4469
Jul 27 20:58:37.476967 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:37.476927    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" event={"ID":"758a99e3-bc0c-48b1-890b-fd6f037e37d3","Type":"ContainerStarted","Data":"749d4f3f361ac172d0d86ddff209b4cb25df1182b8112965c86ad0f986ba4469"}
Jul 27 20:58:38.409729 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:38.409698    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h6fb9"
Jul 27 20:58:41.493391 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:41.493313    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" event={"ID":"758a99e3-bc0c-48b1-890b-fd6f037e37d3","Type":"ContainerStarted","Data":"3772947b1cdc3761fab6251d0c5ae5d02b4e7fadb18524a3d9450cd5414edd1a"}
Jul 27 20:58:41.493391 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:41.493359    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" event={"ID":"758a99e3-bc0c-48b1-890b-fd6f037e37d3","Type":"ContainerStarted","Data":"5b63208de37b0b47f641d155ce21dcac4b79cf33e13cbdc48f5deb59b36dbac7"}
Jul 27 20:58:41.493391 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:41.493373    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" event={"ID":"758a99e3-bc0c-48b1-890b-fd6f037e37d3","Type":"ContainerStarted","Data":"5335a4c794189d78381bf41441c4e994e6b7ec40fbec212faf65603479e91669"}
Jul 27 20:58:41.516851 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:41.516763    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" podStartSLOduration=1.4309391329999999 podStartE2EDuration="5.516746378s" totalImagesPullingTime="4.085807245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:41.511556014 +0000 UTC m=+64.814800778" watchObservedRunningTime="2026-07-27 20:58:41.516746378 +0000 UTC m=+64.819991142"
Jul 27 20:58:42.855904 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:42.855866    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:58:42.858520 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:42.858502    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 27 20:58:42.866546 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:42.866527    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 20:58:42.866604 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:42.866584    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 20:59:46.866568952 +0000 UTC m=+130.169813695 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : secret "metrics-daemon-secret" not found
Jul 27 20:58:42.956398 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:42.956368    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:42.958704 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:42.958686    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 27 20:58:42.969727 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:42.969703    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3851429a-b5b9-4a29-b124-f84984721273-original-pull-secret\") pod \"global-pull-secret-syncer-dp2gb\" (UID: \"3851429a-b5b9-4a29-b124-f84984721273\") " pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:43.056828 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.056800    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:43.059393 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.059373    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 27 20:58:43.069441 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.069418    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 27 20:58:43.079749 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.079722    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrckv\" (UniqueName: \"kubernetes.io/projected/2ee93eb9-66cf-4adb-b494-72fd0d8d220c-kube-api-access-lrckv\") pod \"network-check-target-4x6mt\" (UID: \"2ee93eb9-66cf-4adb-b494-72fd0d8d220c\") " pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:43.106059 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.106008    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hn7dl\""
Jul 27 20:58:43.114324 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.114303    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:43.209513 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.209485    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dp2gb"
Jul 27 20:58:43.251596 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.251555    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4x6mt"]
Jul 27 20:58:43.255199 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:58:43.255166    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee93eb9_66cf_4adb_b494_72fd0d8d220c.slice/crio-a865a0609743c5e7e70e6d65c8257aa5d11627dbcbdc3a9a7779fdfd2aeda014 WatchSource:0}: Error finding container a865a0609743c5e7e70e6d65c8257aa5d11627dbcbdc3a9a7779fdfd2aeda014: Status 404 returned error can't find the container with id a865a0609743c5e7e70e6d65c8257aa5d11627dbcbdc3a9a7779fdfd2aeda014
Jul 27 20:58:43.330321 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.330295    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dp2gb"]
Jul 27 20:58:43.333395 ip-10-0-140-33 kubenswrapper[2552]: W0727 20:58:43.333365    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3851429a_b5b9_4a29_b124_f84984721273.slice/crio-a0e889c9bb829524d14c60b0092182e18496e1685209c1cd7561e92ea5b8f910 WatchSource:0}: Error finding container a0e889c9bb829524d14c60b0092182e18496e1685209c1cd7561e92ea5b8f910: Status 404 returned error can't find the container with id a0e889c9bb829524d14c60b0092182e18496e1685209c1cd7561e92ea5b8f910
Jul 27 20:58:43.498758 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.498718    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dp2gb" event={"ID":"3851429a-b5b9-4a29-b124-f84984721273","Type":"ContainerStarted","Data":"a0e889c9bb829524d14c60b0092182e18496e1685209c1cd7561e92ea5b8f910"}
Jul 27 20:58:43.499879 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.499855    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4x6mt" event={"ID":"2ee93eb9-66cf-4adb-b494-72fd0d8d220c","Type":"ContainerStarted","Data":"a865a0609743c5e7e70e6d65c8257aa5d11627dbcbdc3a9a7779fdfd2aeda014"}
Jul 27 20:58:43.963293 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:43.963251    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:58:43.963742 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:43.963408    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:58:43.963742 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:58:43.963495    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 20:59:15.963474124 +0000 UTC m=+99.266718890 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:58:48.518092 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:48.518058    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4x6mt" event={"ID":"2ee93eb9-66cf-4adb-b494-72fd0d8d220c","Type":"ContainerStarted","Data":"7e1784ac47f4687d3ed967e61c2805597ced3e7205fce7c9c951bf2b860b478c"}
Jul 27 20:58:48.518552 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:48.518119    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:58:48.519438 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:48.519411    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dp2gb" event={"ID":"3851429a-b5b9-4a29-b124-f84984721273","Type":"ContainerStarted","Data":"ece9b2c003e07d7f20397522ffabc3ca11738d84b681f2d20519d39d30ac4e12"}
Jul 27 20:58:48.532784 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:48.532743    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4x6mt" podStartSLOduration=67.043969582 podStartE2EDuration="1m11.532729305s" totalImagesPullingTime="4.488759723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:48.531736416 +0000 UTC m=+71.834981181" watchObservedRunningTime="2026-07-27 20:58:48.532729305 +0000 UTC m=+71.835974048"
Jul 27 20:58:48.545312 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:58:48.545269    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dp2gb" podStartSLOduration=66.128646789 podStartE2EDuration="1m10.545254014s" totalImagesPullingTime="4.416607225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 20:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 20:58:48.544936185 +0000 UTC m=+71.848180956" watchObservedRunningTime="2026-07-27 20:58:48.545254014 +0000 UTC m=+71.848498826"
Jul 27 20:59:15.975141 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:59:15.974991    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 20:59:15.975141 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:59:15.975132    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 20:59:15.975764 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:59:15.975207    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 21:00:19.975187379 +0000 UTC m=+163.278432145 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 20:59:19.524103 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:59:19.524079    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4x6mt"
Jul 27 20:59:46.886497 ip-10-0-140-33 kubenswrapper[2552]: I0727 20:59:46.886465    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 20:59:46.886987 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:59:46.886566    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 20:59:46.886987 ip-10-0-140-33 kubenswrapper[2552]: E0727 20:59:46.886632    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 21:01:48.886618242 +0000 UTC m=+252.189862985 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : secret "metrics-daemon-secret" not found
Jul 27 21:00:15.078549 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:00:15.078509    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mfplf" podUID="a5b0321d-1d87-409a-9277-a7002c2f80b8"
Jul 27 21:00:15.210267 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:00:15.210213    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 21:00:15.760580 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:00:15.760553    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mfplf"
Jul 27 21:00:20.006445 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:00:20.006398    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 21:00:20.006880 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:00:20.006551    2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 27 21:00:20.006880 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:00:20.006633    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls podName:a5b0321d-1d87-409a-9277-a7002c2f80b8 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:22.006617027 +0000 UTC m=+285.309861770 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls") pod "dns-default-mfplf" (UID: "a5b0321d-1d87-409a-9277-a7002c2f80b8") : secret "dns-default-metrics-tls" not found
Jul 27 21:00:30.191881 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:00:30.191805    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 21:01:48.968817 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:48.968771    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 21:01:48.969330 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:48.968922    2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 27 21:01:48.969330 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:48.968988    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs podName:0d1c14b4-7aec-4c81-98c5-55841ddfa641 nodeName:}" failed. No retries permitted until 2026-07-27 21:03:50.968970336 +0000 UTC m=+374.272215078 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs") pod "network-metrics-daemon-cgq4s" (UID: "0d1c14b4-7aec-4c81-98c5-55841ddfa641") : secret "metrics-daemon-secret" not found
Jul 27 21:01:58.099901 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.099822    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f998m"]
Jul 27 21:01:58.101917 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.101901    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.104958 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.104938    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 27 21:01:58.104958 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.104954    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 27 21:01:58.105543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.105527    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 27 21:01:58.105599 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.105532    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 27 21:01:58.106236 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.106207    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g6v4k\""
Jul 27 21:01:58.118552 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.118522    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f998m"]
Jul 27 21:01:58.227944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.227911    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qwhq\" (UniqueName: \"kubernetes.io/projected/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-api-access-4qwhq\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.227944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.227950    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.228163 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.227968    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.228163 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.228007    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-crio-socket\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.228163 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.228028    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-data-volume\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329001 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.328965    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qwhq\" (UniqueName: \"kubernetes.io/projected/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-api-access-4qwhq\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329001 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.329002    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.329027    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.329074    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-crio-socket\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.329095    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.329097    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-data-volume\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.329191    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:01:58.829169836 +0000 UTC m=+262.132414599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : configmap "kube-rbac-proxy" not found
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.329197    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:01:58.329265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.329247    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-crio-socket\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.329551 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.329258    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:01:58.829246088 +0000 UTC m=+262.132490838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:01:58.329551 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.329407    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-data-volume\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.339282 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.339264    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qwhq\" (UniqueName: \"kubernetes.io/projected/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-api-access-4qwhq\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.832111 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.832069    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.832111 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:58.832115    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:58.832346 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.832187    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:01:58.832346 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.832205    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:01:58.832346 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.832270    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:01:59.832252432 +0000 UTC m=+263.135497184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : configmap "kube-rbac-proxy" not found
Jul 27 21:01:58.832445 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:58.832350    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:01:59.832334315 +0000 UTC m=+263.135579065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:01:59.838680 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.838649    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:59.838680 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.838682    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:01:59.839122 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:59.838770    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:01:59.839122 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:59.838786    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:01:59.839122 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:59.838832    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:01.838817019 +0000 UTC m=+265.142061762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : configmap "kube-rbac-proxy" not found
Jul 27 21:01:59.839122 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:01:59.838846    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:01.838839738 +0000 UTC m=+265.142084481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:01:59.887051 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.887020    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dc22d_e2d0da7f-e17f-4522-bc6d-9d57c51f2d06/dns-node-resolver/0.log"
Jul 27 21:01:59.961010 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.960979    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"]
Jul 27 21:01:59.963210 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.963196    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"
Jul 27 21:01:59.966182 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.966161    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-f8fsm\""
Jul 27 21:01:59.966323 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.966161    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 27 21:01:59.967017 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.967000    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 27 21:01:59.973452 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:01:59.973434    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"]
Jul 27 21:02:00.040585 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:00.040554    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbsww\" (UniqueName: \"kubernetes.io/projected/cafab56b-d0f9-4ae1-871f-578693aafde7-kube-api-access-bbsww\") pod \"migrator-6865c9967-rvks2\" (UID: \"cafab56b-d0f9-4ae1-871f-578693aafde7\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"
Jul 27 21:02:00.141638 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:00.141556    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbsww\" (UniqueName: \"kubernetes.io/projected/cafab56b-d0f9-4ae1-871f-578693aafde7-kube-api-access-bbsww\") pod \"migrator-6865c9967-rvks2\" (UID: \"cafab56b-d0f9-4ae1-871f-578693aafde7\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"
Jul 27 21:02:00.149387 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:00.149362    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbsww\" (UniqueName: \"kubernetes.io/projected/cafab56b-d0f9-4ae1-871f-578693aafde7-kube-api-access-bbsww\") pod \"migrator-6865c9967-rvks2\" (UID: \"cafab56b-d0f9-4ae1-871f-578693aafde7\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"
Jul 27 21:02:00.271763 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:00.271721    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"
Jul 27 21:02:00.387943 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:00.387830    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2"]
Jul 27 21:02:00.390412 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:00.390377    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcafab56b_d0f9_4ae1_871f_578693aafde7.slice/crio-4214f57aaa01457a5d2ff4f88c201e89c5520ab96280897c2834cb783a55292c WatchSource:0}: Error finding container 4214f57aaa01457a5d2ff4f88c201e89c5520ab96280897c2834cb783a55292c: Status 404 returned error can't find the container with id 4214f57aaa01457a5d2ff4f88c201e89c5520ab96280897c2834cb783a55292c
Jul 27 21:02:00.886912 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:00.886890    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qnt7r_93ced636-3c9e-428d-80e8-acb5ea71c7d2/node-ca/0.log"
Jul 27 21:02:01.049566 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:01.049534    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2" event={"ID":"cafab56b-d0f9-4ae1-871f-578693aafde7","Type":"ContainerStarted","Data":"4214f57aaa01457a5d2ff4f88c201e89c5520ab96280897c2834cb783a55292c"}
Jul 27 21:02:01.854601 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:01.854578    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:01.854694 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:01.854608    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:01.854749 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:01.854694    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:02:01.854749 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:01.854710    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:02:01.854749 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:01.854740    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:05.854724271 +0000 UTC m=+269.157969014 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : configmap "kube-rbac-proxy" not found
Jul 27 21:02:01.854863 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:01.854785    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:05.854778996 +0000 UTC m=+269.158023738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:02:01.886878 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:01.886859    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bkp2g_7c8704a1-3d26-44a4-bb4c-0659e2702945/serve-healthcheck-canary/0.log"
Jul 27 21:02:02.054679 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:02.054595    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2" event={"ID":"cafab56b-d0f9-4ae1-871f-578693aafde7","Type":"ContainerStarted","Data":"4eb5236774dc70cdd6f22bd9ca48df89c08646d6d5663a612ddbbcb83839a5dc"}
Jul 27 21:02:02.054679 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:02.054651    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2" event={"ID":"cafab56b-d0f9-4ae1-871f-578693aafde7","Type":"ContainerStarted","Data":"fb9fffeaae885c74eaf89f42e65ecea25d6332c9f1dd98a0290ad66133e8ac1c"}
Jul 27 21:02:02.084141 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:02.084097    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-rvks2" podStartSLOduration=1.752234465 podStartE2EDuration="3.084082207s" totalImagesPullingTime="1.331847742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:02.082546345 +0000 UTC m=+265.385791110" watchObservedRunningTime="2026-07-27 21:02:02.084082207 +0000 UTC m=+265.387326972"
Jul 27 21:02:05.884414 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:05.884369    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:05.884414 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:05.884416    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:05.884874 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:05.884489    2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 27 21:02:05.884874 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:05.884568    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:13.884553622 +0000 UTC m=+277.187798366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : configmap "kube-rbac-proxy" not found
Jul 27 21:02:05.884874 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:05.884571    2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 27 21:02:05.884874 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:05.884687    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls podName:67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:13.884675665 +0000 UTC m=+277.187920409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f998m" (UID: "67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2") : secret "insights-runtime-extractor-tls" not found
Jul 27 21:02:13.942737 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:13.942700    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:13.942737 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:13.942736    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:13.943369 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:13.943348    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:13.945093 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:13.945073    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f998m\" (UID: \"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2\") " pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:14.010510 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:14.010484    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f998m"
Jul 27 21:02:14.134371 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:14.134343    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f998m"]
Jul 27 21:02:14.137742 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:14.137715    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67afd2fe_2c35_4d49_bfe5_bdd53a9a85b2.slice/crio-acda103fb9abd6eada024376ba2d6e12ff18f0e930385fa634566c1a2090be07 WatchSource:0}: Error finding container acda103fb9abd6eada024376ba2d6e12ff18f0e930385fa634566c1a2090be07: Status 404 returned error can't find the container with id acda103fb9abd6eada024376ba2d6e12ff18f0e930385fa634566c1a2090be07
Jul 27 21:02:15.093285 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:15.093253    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f998m" event={"ID":"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2","Type":"ContainerStarted","Data":"acda103fb9abd6eada024376ba2d6e12ff18f0e930385fa634566c1a2090be07"}
Jul 27 21:02:18.761377 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:18.761329    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mfplf" podUID="a5b0321d-1d87-409a-9277-a7002c2f80b8"
Jul 27 21:02:19.106565 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:19.106543    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mfplf"
Jul 27 21:02:20.111878 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:20.111838    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f998m" event={"ID":"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2","Type":"ContainerStarted","Data":"70a172ff82b1d023fe81cc0bea5e445849a5ed6531129610d9f9249ec0dd36c2"}
Jul 27 21:02:22.007560 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:22.007526    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 21:02:22.010440 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:22.010403    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a5b0321d-1d87-409a-9277-a7002c2f80b8-metrics-tls\") pod \"dns-default-mfplf\" (UID: \"a5b0321d-1d87-409a-9277-a7002c2f80b8\") " pod="openshift-dns/dns-default-mfplf"
Jul 27 21:02:22.110121 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:22.110086    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bvklp\""
Jul 27 21:02:22.118206 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:22.118007    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mfplf"
Jul 27 21:02:22.613891 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:22.613865    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mfplf"]
Jul 27 21:02:22.617011 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:22.616986    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b0321d_1d87_409a_9277_a7002c2f80b8.slice/crio-24379543d9927599240b7abb41c727fd2bf0fe28e0d7b6539bf1c66d9ce65d66 WatchSource:0}: Error finding container 24379543d9927599240b7abb41c727fd2bf0fe28e0d7b6539bf1c66d9ce65d66: Status 404 returned error can't find the container with id 24379543d9927599240b7abb41c727fd2bf0fe28e0d7b6539bf1c66d9ce65d66
Jul 27 21:02:23.127198 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:23.127118    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f998m" event={"ID":"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2","Type":"ContainerStarted","Data":"60ff6e5b0e5f7abd08e25e59ddf73a6908756f40ef0634bc750aa4fdebce9aa2"}
Jul 27 21:02:23.129162 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:23.129133    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mfplf" event={"ID":"a5b0321d-1d87-409a-9277-a7002c2f80b8","Type":"ContainerStarted","Data":"24379543d9927599240b7abb41c727fd2bf0fe28e0d7b6539bf1c66d9ce65d66"}
Jul 27 21:02:25.138206 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.138163    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mfplf" event={"ID":"a5b0321d-1d87-409a-9277-a7002c2f80b8","Type":"ContainerStarted","Data":"fca50542f766d68a69759ba7b68641d8260ef2791219a802312bfc0e5801e433"}
Jul 27 21:02:25.138607 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.138288    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mfplf"
Jul 27 21:02:25.138607 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.138314    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mfplf" event={"ID":"a5b0321d-1d87-409a-9277-a7002c2f80b8","Type":"ContainerStarted","Data":"512feee13c58e7b3ebae0bd2c16c510414bcb572e4ef39d82630201e38cff4e8"}
Jul 27 21:02:25.159157 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.159108    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mfplf" podStartSLOduration=251.561555095 podStartE2EDuration="4m13.159093452s" totalImagesPullingTime="1.597538357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:25.158680922 +0000 UTC m=+288.461925687" watchObservedRunningTime="2026-07-27 21:02:25.159093452 +0000 UTC m=+288.462338217"
Jul 27 21:02:25.899594 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.899559    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"]
Jul 27 21:02:25.918028 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.917915    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:25.920339 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.920312    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"]
Jul 27 21:02:25.920614 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.920593    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 27 21:02:25.921052 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.921029    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 27 21:02:25.921166 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.921056    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 27 21:02:25.921677 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.921613    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 27 21:02:25.922269 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.922125    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 27 21:02:25.922639 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:25.922619    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-xdcl5\""
Jul 27 21:02:26.040577 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.040533    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/630071f6-e860-4ba1-88c4-797f5c74c00a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.040734 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.040665    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/630071f6-e860-4ba1-88c4-797f5c74c00a-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.040734 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.040703    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/630071f6-e860-4ba1-88c4-797f5c74c00a-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.040835 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.040743    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq9jk\" (UniqueName: \"kubernetes.io/projected/630071f6-e860-4ba1-88c4-797f5c74c00a-kube-api-access-zq9jk\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.141667 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.141631    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq9jk\" (UniqueName: \"kubernetes.io/projected/630071f6-e860-4ba1-88c4-797f5c74c00a-kube-api-access-zq9jk\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.142123 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.141677    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/630071f6-e860-4ba1-88c4-797f5c74c00a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.142123 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.141741    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/630071f6-e860-4ba1-88c4-797f5c74c00a-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.142123 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.141762    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/630071f6-e860-4ba1-88c4-797f5c74c00a-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.142606 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.142583    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/630071f6-e860-4ba1-88c4-797f5c74c00a-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.144543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.144492    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/630071f6-e860-4ba1-88c4-797f5c74c00a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.144689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.144670    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/630071f6-e860-4ba1-88c4-797f5c74c00a-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.151242 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.151169    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq9jk\" (UniqueName: \"kubernetes.io/projected/630071f6-e860-4ba1-88c4-797f5c74c00a-kube-api-access-zq9jk\") pod \"prometheus-operator-84d69f794d-7fx4h\" (UID: \"630071f6-e860-4ba1-88c4-797f5c74c00a\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.230448 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.230411    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"
Jul 27 21:02:26.358734 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:26.358710    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-7fx4h"]
Jul 27 21:02:26.361340 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:26.361319    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630071f6_e860_4ba1_88c4_797f5c74c00a.slice/crio-ba6614f8eed1efce7fbf4b5a69fd837178b769392e50569deabf7b6ba41df251 WatchSource:0}: Error finding container ba6614f8eed1efce7fbf4b5a69fd837178b769392e50569deabf7b6ba41df251: Status 404 returned error can't find the container with id ba6614f8eed1efce7fbf4b5a69fd837178b769392e50569deabf7b6ba41df251
Jul 27 21:02:27.145943 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:27.145899    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h" event={"ID":"630071f6-e860-4ba1-88c4-797f5c74c00a","Type":"ContainerStarted","Data":"ba6614f8eed1efce7fbf4b5a69fd837178b769392e50569deabf7b6ba41df251"}
Jul 27 21:02:27.148236 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:27.148193    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f998m" event={"ID":"67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2","Type":"ContainerStarted","Data":"f809556fff3e1e7fd817b59cf272c704c65c91641869c858fbae197e5b867afe"}
Jul 27 21:02:27.166149 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:27.166101    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f998m" podStartSLOduration=17.103144563 podStartE2EDuration="29.166085643s" totalImagesPullingTime="12.06294108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:01:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:27.165423805 +0000 UTC m=+290.468668574" watchObservedRunningTime="2026-07-27 21:02:27.166085643 +0000 UTC m=+290.469330408"
Jul 27 21:02:28.153487 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:28.153397    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h" event={"ID":"630071f6-e860-4ba1-88c4-797f5c74c00a","Type":"ContainerStarted","Data":"5447ae72361de7c7b67c0a253268dac450a8172e8a198108b7ab5ec335f08a98"}
Jul 27 21:02:28.153487 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:28.153479    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h" event={"ID":"630071f6-e860-4ba1-88c4-797f5c74c00a","Type":"ContainerStarted","Data":"f57792a853e725982dca82957d5cde1f67953a7d57fcb7ff6c0d968be942395a"}
Jul 27 21:02:28.169525 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:28.169480    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-7fx4h" podStartSLOduration=1.650638432 podStartE2EDuration="3.169467574s" totalImagesPullingTime="1.518829142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:28.168208194 +0000 UTC m=+291.471452963" watchObservedRunningTime="2026-07-27 21:02:28.169467574 +0000 UTC m=+291.472712338"
Jul 27 21:02:30.217559 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.217530    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"]
Jul 27 21:02:30.220257 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.220242    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.223179 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.223149    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2bzwv\""
Jul 27 21:02:30.223331 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.223149    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 27 21:02:30.223805 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.223783    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 27 21:02:30.223964 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.223893    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 27 21:02:30.224891 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.224873    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wlbcw"]
Jul 27 21:02:30.227391 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.227375    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.229728 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.229713    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 27 21:02:30.229848 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.229834    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-28jhl\""
Jul 27 21:02:30.230163 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.230150    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 27 21:02:30.231328 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.231313    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 27 21:02:30.233631 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.233611    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"]
Jul 27 21:02:30.370862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.370829    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-tls\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371045 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.370867    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-wtmp\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371045 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.370895    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzd66\" (UniqueName: \"kubernetes.io/projected/c0e37be3-390d-4583-aed3-bde7e9f16223-kube-api-access-gzd66\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371045 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.370985    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-accelerators-collector-config\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371045 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371026    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371108    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371142    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-root\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371163    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371191    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.371463 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371263    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0e37be3-390d-4583-aed3-bde7e9f16223-metrics-client-ca\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371463 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371298    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r8kn\" (UniqueName: \"kubernetes.io/projected/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-api-access-7r8kn\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.371463 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371321    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.371463 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371420    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.371463 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371457    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-sys\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.371626 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.371482    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-textfile\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.472862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472782    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-root\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.472862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472814    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.472862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472835    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.472862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472860    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0e37be3-390d-4583-aed3-bde7e9f16223-metrics-client-ca\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472921    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-root\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472960    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r8kn\" (UniqueName: \"kubernetes.io/projected/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-api-access-7r8kn\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.472999    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473025    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473056    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-sys\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473080    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-textfile\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473134    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-tls\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473157    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-wtmp\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473181    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzd66\" (UniqueName: \"kubernetes.io/projected/c0e37be3-390d-4583-aed3-bde7e9f16223-kube-api-access-gzd66\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473219    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-accelerators-collector-config\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:30.473329    2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473359    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-sys\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:30.473405    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-tls podName:c0e37be3-390d-4583-aed3-bde7e9f16223 nodeName:}" failed. No retries permitted until 2026-07-27 21:02:30.973383829 +0000 UTC m=+294.276628584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-tls") pod "node-exporter-wlbcw" (UID: "c0e37be3-390d-4583-aed3-bde7e9f16223") : secret "node-exporter-tls" not found
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473440    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.473507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473481    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473899 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473530    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-wtmp\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473899 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473544    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0e37be3-390d-4583-aed3-bde7e9f16223-metrics-client-ca\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473899 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473772    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-accelerators-collector-config\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.473899 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473871    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-textfile\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.474091 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473931    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.474091 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.473994    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.474394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.474372    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.475729 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.475705    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.475824 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.475808    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.475890 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.475874    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.483409 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.483390    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r8kn\" (UniqueName: \"kubernetes.io/projected/2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6-kube-api-access-7r8kn\") pod \"kube-state-metrics-56d9f944b4-877nb\" (UID: \"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.483491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.483414    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzd66\" (UniqueName: \"kubernetes.io/projected/c0e37be3-390d-4583-aed3-bde7e9f16223-kube-api-access-gzd66\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.529637 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.529611    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"
Jul 27 21:02:30.654848 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.654669    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-877nb"]
Jul 27 21:02:30.657136 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:30.657112    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bed2be2_fcfa_4b1a_9f50_2bc6acfa28f6.slice/crio-b50948a8997b335fb8192bed5e649142a3d47a935d22afbe9371b639915cba4b WatchSource:0}: Error finding container b50948a8997b335fb8192bed5e649142a3d47a935d22afbe9371b639915cba4b: Status 404 returned error can't find the container with id b50948a8997b335fb8192bed5e649142a3d47a935d22afbe9371b639915cba4b
Jul 27 21:02:30.978157 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.978124    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-tls\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:30.980485 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:30.980459    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0e37be3-390d-4583-aed3-bde7e9f16223-node-exporter-tls\") pod \"node-exporter-wlbcw\" (UID: \"c0e37be3-390d-4583-aed3-bde7e9f16223\") " pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:31.137082 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:31.137048    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wlbcw"
Jul 27 21:02:31.146745 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:31.146719    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e37be3_390d_4583_aed3_bde7e9f16223.slice/crio-1fdfa276745795470f547c6d5ab5a9f06ed77ed075a54da729b0539a7e1bbaba WatchSource:0}: Error finding container 1fdfa276745795470f547c6d5ab5a9f06ed77ed075a54da729b0539a7e1bbaba: Status 404 returned error can't find the container with id 1fdfa276745795470f547c6d5ab5a9f06ed77ed075a54da729b0539a7e1bbaba
Jul 27 21:02:31.164697 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:31.164672    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wlbcw" event={"ID":"c0e37be3-390d-4583-aed3-bde7e9f16223","Type":"ContainerStarted","Data":"1fdfa276745795470f547c6d5ab5a9f06ed77ed075a54da729b0539a7e1bbaba"}
Jul 27 21:02:31.165807 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:31.165784    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb" event={"ID":"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6","Type":"ContainerStarted","Data":"b50948a8997b335fb8192bed5e649142a3d47a935d22afbe9371b639915cba4b"}
Jul 27 21:02:33.174780 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:33.174746    2552 generic.go:356] "Generic (PLEG): container finished" podID="c0e37be3-390d-4583-aed3-bde7e9f16223" containerID="01da23a4b61327c1a35f3d79faec9c66417bceb896cc85fe2c8dd5c2cd5b89be" exitCode=0
Jul 27 21:02:33.175267 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:33.174842    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wlbcw" event={"ID":"c0e37be3-390d-4583-aed3-bde7e9f16223","Type":"ContainerDied","Data":"01da23a4b61327c1a35f3d79faec9c66417bceb896cc85fe2c8dd5c2cd5b89be"}
Jul 27 21:02:33.176968 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:33.176948    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb" event={"ID":"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6","Type":"ContainerStarted","Data":"b8297b16518f2d81d2f364d7f5a7a847dd8dffac9bf526c3e9edb9e5af505a9a"}
Jul 27 21:02:33.177093 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:33.176990    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb" event={"ID":"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6","Type":"ContainerStarted","Data":"97fb599885c5bed1346b2c02f66185ac488e014d77e4a9c29563616756eee02a"}
Jul 27 21:02:33.177093 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:33.177011    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb" event={"ID":"2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6","Type":"ContainerStarted","Data":"0d1437288a92171eaab1d5a08218ed4cbd4e23cc05e3323fa21dd320fa70a91a"}
Jul 27 21:02:33.193244 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:02:33.193195    2552 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-cgq4s" podUID="0d1c14b4-7aec-4c81-98c5-55841ddfa641"
Jul 27 21:02:33.210715 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:33.210673    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-877nb" podStartSLOduration=1.5745728190000001 podStartE2EDuration="3.210659017s" totalImagesPullingTime="1.636086198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:33.210104767 +0000 UTC m=+296.513349533" watchObservedRunningTime="2026-07-27 21:02:33.210659017 +0000 UTC m=+296.513903783"
Jul 27 21:02:34.182083 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.182043    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wlbcw" event={"ID":"c0e37be3-390d-4583-aed3-bde7e9f16223","Type":"ContainerStarted","Data":"52ae8b567b01e168528a5457232bd4b8d97fb0ebe2b48a69f915bc2994ebba70"}
Jul 27 21:02:34.182581 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.182101    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wlbcw" event={"ID":"c0e37be3-390d-4583-aed3-bde7e9f16223","Type":"ContainerStarted","Data":"713d95b2832f79919cc90246d337b71271f4485d0e456f54940774f870a0ce25"}
Jul 27 21:02:34.199466 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.199419    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wlbcw" podStartSLOduration=2.956296413 podStartE2EDuration="4.199403644s" totalImagesPullingTime="1.147899002s" totalInitContainerRuntime="95.208229ms" isStatefulPod=true podCreationTimestamp="2026-07-27 21:02:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:34.199174264 +0000 UTC m=+297.502419028" watchObservedRunningTime="2026-07-27 21:02:34.199403644 +0000 UTC m=+297.502648410"
Jul 27 21:02:34.503990 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.503919    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76c949c4f5-xbct8"]
Jul 27 21:02:34.507742 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.507721    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.511289 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.511260    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 27 21:02:34.511412 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.511297    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 27 21:02:34.511412 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.511351    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 27 21:02:34.511412 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.511378    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-348nf7u0shuba\""
Jul 27 21:02:34.511412 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.511303    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6p4cv\""
Jul 27 21:02:34.511619 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.511260    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 27 21:02:34.514935 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.514919    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76c949c4f5-xbct8"]
Jul 27 21:02:34.604684 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604651    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/77dfb373-b265-4939-a281-653fba1fb080-metrics-server-audit-profiles\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.604852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604694    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnj8c\" (UniqueName: \"kubernetes.io/projected/77dfb373-b265-4939-a281-653fba1fb080-kube-api-access-jnj8c\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.604852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604723    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-secret-metrics-server-client-certs\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.604852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604764    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-client-ca-bundle\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.604852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604790    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77dfb373-b265-4939-a281-653fba1fb080-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.604852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604816    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-secret-metrics-server-tls\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.604852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.604843    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/77dfb373-b265-4939-a281-653fba1fb080-audit-log\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705665 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705636    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnj8c\" (UniqueName: \"kubernetes.io/projected/77dfb373-b265-4939-a281-653fba1fb080-kube-api-access-jnj8c\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705762 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705671    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-secret-metrics-server-client-certs\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705762 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705710    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-client-ca-bundle\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705762 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705729    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77dfb373-b265-4939-a281-653fba1fb080-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705762 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705745    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-secret-metrics-server-tls\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705999 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705769    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/77dfb373-b265-4939-a281-653fba1fb080-audit-log\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.705999 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.705812    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/77dfb373-b265-4939-a281-653fba1fb080-metrics-server-audit-profiles\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.706362 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.706341    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/77dfb373-b265-4939-a281-653fba1fb080-audit-log\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.706573 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.706553    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77dfb373-b265-4939-a281-653fba1fb080-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.706906 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.706885    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/77dfb373-b265-4939-a281-653fba1fb080-metrics-server-audit-profiles\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.708270 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.708254    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-secret-metrics-server-client-certs\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.708777 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.708759    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-secret-metrics-server-tls\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.708913 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.708894    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/77dfb373-b265-4939-a281-653fba1fb080-client-ca-bundle\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.713696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.713674    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnj8c\" (UniqueName: \"kubernetes.io/projected/77dfb373-b265-4939-a281-653fba1fb080-kube-api-access-jnj8c\") pod \"metrics-server-76c949c4f5-xbct8\" (UID: \"77dfb373-b265-4939-a281-653fba1fb080\") " pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.817750 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.817678    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:34.961519 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:34.961481    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76c949c4f5-xbct8"]
Jul 27 21:02:34.965933 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:34.965897    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77dfb373_b265_4939_a281_653fba1fb080.slice/crio-f6548854e1cb273a5d41bd1fde3a785179842a0312395653814eae5dd6953dbd WatchSource:0}: Error finding container f6548854e1cb273a5d41bd1fde3a785179842a0312395653814eae5dd6953dbd: Status 404 returned error can't find the container with id f6548854e1cb273a5d41bd1fde3a785179842a0312395653814eae5dd6953dbd
Jul 27 21:02:35.144139 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:35.144056    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mfplf"
Jul 27 21:02:35.190899 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:35.190864    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8" event={"ID":"77dfb373-b265-4939-a281-653fba1fb080","Type":"ContainerStarted","Data":"f6548854e1cb273a5d41bd1fde3a785179842a0312395653814eae5dd6953dbd"}
Jul 27 21:02:36.423857 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.423817    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:02:36.426753 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.426732    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.435110 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.434909    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 21:02:36.435412 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.435309    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 21:02:36.435615 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.435597    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 21:02:36.436018 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.435774    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 21:02:36.436018 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.435826    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 21:02:36.436018 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.435898    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 21:02:36.436018 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.435827    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 21:02:36.436386 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.436189    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 21:02:36.436386 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.436245    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 21:02:36.436386 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.436193    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 21:02:36.436598 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.436189    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 21:02:36.436663 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.436640    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4kn4w\""
Jul 27 21:02:36.436760 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.436722    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-632repka64g4l\""
Jul 27 21:02:36.437031 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.437017    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 21:02:36.437292 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.437271    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 21:02:36.437926 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.437908    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 21:02:36.456269 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.456163    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:02:36.519997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.519970    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.519997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.519998    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520032    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520055    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520104    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-config\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520120    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520144    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520164    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520177    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520195    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520219    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520315    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520351    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520376    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520400    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520428    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520453    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520483    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqt75\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-kube-api-access-gqt75\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520520    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.520688 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.520562    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.621927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.621890    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.621927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.621925    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.621947    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-config\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.621963    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.621981    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622041    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622088    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622120    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622148    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622192    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622243    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622267    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622292    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622323    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622347    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622372    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqt75\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-kube-api-access-gqt75\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622403    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622456    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622498    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.622537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622521    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.623212 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.622793    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.623783 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.623755    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.623902 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.623814    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.623959 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.623924    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.624076 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.624053    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.624918 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.624895    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.626471 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.626436    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.627529 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.627497    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.630143 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.628449    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.630311 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.630193    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.630616 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.630594    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-config-out\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.630694 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.630642    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.631058 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.631017    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-config\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.631148 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.631055    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.631148 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.631096    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-web-config\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.631612 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.631574    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.631764 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.631741    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.631764 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.631764    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.633777 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.633747    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.634010 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.633989    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqt75\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-kube-api-access-gqt75\") pod \"prometheus-k8s-0\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.738950 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.738862    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:36.899910 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.899798    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:02:36.904920 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:02:36.904895    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13ca80bb_d95a_45be_b816_731b9f451e4c.slice/crio-745c47aa4288fae93e257da035b97a488b71521973965eb970e4a6fbf39bd71c WatchSource:0}: Error finding container 745c47aa4288fae93e257da035b97a488b71521973965eb970e4a6fbf39bd71c: Status 404 returned error can't find the container with id 745c47aa4288fae93e257da035b97a488b71521973965eb970e4a6fbf39bd71c
Jul 27 21:02:36.999983 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:36.999900    2552 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" podUID="758a99e3-bc0c-48b1-890b-fd6f037e37d3" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 21:02:37.074821 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:37.074797    2552 kubelet.go:1648] "Image garbage collection succeeded"
Jul 27 21:02:37.201292 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:37.201112    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8" event={"ID":"77dfb373-b265-4939-a281-653fba1fb080","Type":"ContainerStarted","Data":"7b2e8c710747d906d4e6dfed70e6d286e6a7eaffdfb5245373082dd395b23926"}
Jul 27 21:02:37.227557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:37.202160    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"745c47aa4288fae93e257da035b97a488b71521973965eb970e4a6fbf39bd71c"}
Jul 27 21:02:37.227557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:37.221362    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8" podStartSLOduration=1.720660775 podStartE2EDuration="3.22134818s" totalImagesPullingTime="1.500687405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:37.220908752 +0000 UTC m=+300.524153516" watchObservedRunningTime="2026-07-27 21:02:37.22134818 +0000 UTC m=+300.524592944"
Jul 27 21:02:38.208120 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:38.208088    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80" exitCode=0
Jul 27 21:02:38.208562 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:38.208188    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80"}
Jul 27 21:02:38.209303 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:38.209289    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:02:42.226839 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:42.226806    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7"}
Jul 27 21:02:42.227194 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:42.226881    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c"}
Jul 27 21:02:44.239556 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:44.239512    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478"}
Jul 27 21:02:44.239973 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:44.239669    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c"}
Jul 27 21:02:44.239973 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:44.239719    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88"}
Jul 27 21:02:44.239973 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:44.239762    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerStarted","Data":"f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804"}
Jul 27 21:02:44.274641 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:44.274576    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.075806701 podStartE2EDuration="8.274559008s" totalImagesPullingTime="6.180335512s" totalInitContainerRuntime="18.416795ms" isStatefulPod=false podCreationTimestamp="2026-07-27 21:02:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:02:44.273161456 +0000 UTC m=+307.576406242" watchObservedRunningTime="2026-07-27 21:02:44.274559008 +0000 UTC m=+307.577803777"
Jul 27 21:02:45.192142 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:45.192106    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 21:02:46.739523 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:46.739484    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:02:47.000507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:47.000429    2552 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" podUID="758a99e3-bc0c-48b1-890b-fd6f037e37d3" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 21:02:54.817793 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:54.817757    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:54.818302 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:54.817820    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:02:57.000084 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.000045    2552 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" podUID="758a99e3-bc0c-48b1-890b-fd6f037e37d3" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 27 21:02:57.000462 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.000120    2552 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5"
Jul 27 21:02:57.000590 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.000560    2552 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"3772947b1cdc3761fab6251d0c5ae5d02b4e7fadb18524a3d9450cd5414edd1a"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 27 21:02:57.000625 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.000611    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" podUID="758a99e3-bc0c-48b1-890b-fd6f037e37d3" containerName="service-proxy" containerID="cri-o://3772947b1cdc3761fab6251d0c5ae5d02b4e7fadb18524a3d9450cd5414edd1a" gracePeriod=30
Jul 27 21:02:57.285377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.285301    2552 generic.go:356] "Generic (PLEG): container finished" podID="758a99e3-bc0c-48b1-890b-fd6f037e37d3" containerID="3772947b1cdc3761fab6251d0c5ae5d02b4e7fadb18524a3d9450cd5414edd1a" exitCode=2
Jul 27 21:02:57.285536 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.285374    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" event={"ID":"758a99e3-bc0c-48b1-890b-fd6f037e37d3","Type":"ContainerDied","Data":"3772947b1cdc3761fab6251d0c5ae5d02b4e7fadb18524a3d9450cd5414edd1a"}
Jul 27 21:02:57.285536 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:02:57.285426    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76478669c5-dcxb5" event={"ID":"758a99e3-bc0c-48b1-890b-fd6f037e37d3","Type":"ContainerStarted","Data":"62254739fd10dcec1d4270660b1120db8a6e81579287d276534654fae7f73eb6"}
Jul 27 21:03:14.823503 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:14.823473    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:03:14.827336 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:14.827314    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76c949c4f5-xbct8"
Jul 27 21:03:36.739279 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:36.739176    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:36.758465 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:36.758435    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:37.436136 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:37.436108    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:51.012455 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:51.012419    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 21:03:51.014822 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:51.014805    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d1c14b4-7aec-4c81-98c5-55841ddfa641-metrics-certs\") pod \"network-metrics-daemon-cgq4s\" (UID: \"0d1c14b4-7aec-4c81-98c5-55841ddfa641\") " pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 21:03:51.195739 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:51.195706    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cxsx8\""
Jul 27 21:03:51.203797 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:51.203770    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cgq4s"
Jul 27 21:03:51.329877 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:51.329856    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cgq4s"]
Jul 27 21:03:51.332618 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:03:51.332589    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1c14b4_7aec_4c81_98c5_55841ddfa641.slice/crio-f56119df081a3f27aa011b08dd0b9a11a13f4e13d4d3a536ff337af89514e5d6 WatchSource:0}: Error finding container f56119df081a3f27aa011b08dd0b9a11a13f4e13d4d3a536ff337af89514e5d6: Status 404 returned error can't find the container with id f56119df081a3f27aa011b08dd0b9a11a13f4e13d4d3a536ff337af89514e5d6
Jul 27 21:03:51.468299 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:51.468264    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgq4s" event={"ID":"0d1c14b4-7aec-4c81-98c5-55841ddfa641","Type":"ContainerStarted","Data":"f56119df081a3f27aa011b08dd0b9a11a13f4e13d4d3a536ff337af89514e5d6"}
Jul 27 21:03:53.476829 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:53.476791    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgq4s" event={"ID":"0d1c14b4-7aec-4c81-98c5-55841ddfa641","Type":"ContainerStarted","Data":"deae33a269defaa37f11ab856a22011614e3e98d0947621b4d1c148828495ddd"}
Jul 27 21:03:53.477207 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:53.476857    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cgq4s" event={"ID":"0d1c14b4-7aec-4c81-98c5-55841ddfa641","Type":"ContainerStarted","Data":"8d4ea8500ccb31b39b498b5b02cf1ff1ff849a5ffca72a0ed742310a5dae1862"}
Jul 27 21:03:53.494097 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:53.494043    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cgq4s" podStartSLOduration=375.393719317 podStartE2EDuration="6m16.49402543s" totalImagesPullingTime="1.100306113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 20:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:03:53.492498675 +0000 UTC m=+376.795743440" watchObservedRunningTime="2026-07-27 21:03:53.49402543 +0000 UTC m=+376.797270199"
Jul 27 21:03:54.781735 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.781694    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:03:54.782214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.782081    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="prometheus" containerID="cri-o://6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c" gracePeriod=600
Jul 27 21:03:54.782534 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.782506    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-thanos" containerID="cri-o://168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478" gracePeriod=600
Jul 27 21:03:54.783980 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.782719    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy" containerID="cri-o://d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c" gracePeriod=600
Jul 27 21:03:54.784081 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.782742    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="thanos-sidecar" containerID="cri-o://f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804" gracePeriod=600
Jul 27 21:03:54.784146 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.782755    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="config-reloader" containerID="cri-o://042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7" gracePeriod=600
Jul 27 21:03:54.784348 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:54.782763    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-web" containerID="cri-o://ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88" gracePeriod=600
Jul 27 21:03:55.488242 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488191    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478" exitCode=0
Jul 27 21:03:55.488242 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488215    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c" exitCode=0
Jul 27 21:03:55.488242 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488236    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804" exitCode=0
Jul 27 21:03:55.488242 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488245    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7" exitCode=0
Jul 27 21:03:55.488507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488254    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c" exitCode=0
Jul 27 21:03:55.488507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488257    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478"}
Jul 27 21:03:55.488507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488317    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c"}
Jul 27 21:03:55.488507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488346    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804"}
Jul 27 21:03:55.488507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488370    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7"}
Jul 27 21:03:55.488507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:55.488394    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c"}
Jul 27 21:03:56.023676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.023654    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.048981 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.048959    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-kubelet-serving-ca-bundle\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049092 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049004    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-2\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049092 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049029    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-thanos-prometheus-http-client-file\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049092 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049058    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-kube-rbac-proxy\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049092 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049075    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049092 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049091    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-tls-assets\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049106    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-grpc-tls\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049134    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-metrics-client-certs\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049166    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-trusted-ca-bundle\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049187    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-config\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049210    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-tls\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049257    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-serving-certs-ca-bundle\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049312    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-1\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.049377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049357    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.049756 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049452    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.049756 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.049635    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.050126 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.050103    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.050435 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.050418    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-config-out\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.050602 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.050586    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqt75\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-kube-api-access-gqt75\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.050716 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.050703    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-metrics-client-ca\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.050812 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.050800    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-web-config\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.050918 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.050904    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.051023 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.051010    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-0\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.051149 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.051135    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-db\") pod \"13ca80bb-d95a-45be-b816-731b9f451e4c\" (UID: \"13ca80bb-d95a-45be-b816-731b9f451e4c\") "
Jul 27 21:03:56.051607 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.051591    2552 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.052504 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.052331    2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.052504 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.052353    2552 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.052504 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.052368    2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.052726 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.052597    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:03:56.053597 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.053309    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.053597 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.053383    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.053597 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.053455    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.053597 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.053544    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:03:56.054012 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.053635    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.054692 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.054396    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-config" (OuterVolumeSpecName: "config") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.055175 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.055145    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.056214 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.056184    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.056486 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.056332    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-kube-api-access-gqt75" (OuterVolumeSpecName: "kube-api-access-gqt75") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "kube-api-access-gqt75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:03:56.056486 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.056367    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-config-out" (OuterVolumeSpecName: "config-out") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:03:56.056486 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.056473    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:03:56.056636 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.056558    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.056636 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.056604    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.057531 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.057507    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.072062 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.072038    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-web-config" (OuterVolumeSpecName: "web-config") pod "13ca80bb-d95a-45be-b816-731b9f451e4c" (UID: "13ca80bb-d95a-45be-b816-731b9f451e4c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:03:56.153394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153307    2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-trusted-ca-bundle\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153337    2552 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-config\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153352    2552 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-tls\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153364    2552 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-config-out\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153375    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqt75\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-kube-api-access-gqt75\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153389    2552 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-configmap-metrics-client-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153401    2552 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-web-config\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153414    2552 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153427    2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153437    2552 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/13ca80bb-d95a-45be-b816-731b9f451e4c-prometheus-k8s-db\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153448    2552 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-thanos-prometheus-http-client-file\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153459    2552 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-kube-rbac-proxy\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153470    2552 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153481    2552 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13ca80bb-d95a-45be-b816-731b9f451e4c-tls-assets\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153492    2552 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-grpc-tls\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.153696 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.153503    2552 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/13ca80bb-d95a-45be-b816-731b9f451e4c-secret-metrics-client-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:03:56.495444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.495415    2552 generic.go:356] "Generic (PLEG): container finished" podID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerID="ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88" exitCode=0
Jul 27 21:03:56.495676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.495489    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88"}
Jul 27 21:03:56.495676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.495501    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.495676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.495554    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"13ca80bb-d95a-45be-b816-731b9f451e4c","Type":"ContainerDied","Data":"745c47aa4288fae93e257da035b97a488b71521973965eb970e4a6fbf39bd71c"}
Jul 27 21:03:56.495676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.495592    2552 scope.go:117] "RemoveContainer" containerID="168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478"
Jul 27 21:03:56.504149 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.504128    2552 scope.go:117] "RemoveContainer" containerID="d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c"
Jul 27 21:03:56.510721 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.510707    2552 scope.go:117] "RemoveContainer" containerID="ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88"
Jul 27 21:03:56.516937 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.516902    2552 scope.go:117] "RemoveContainer" containerID="f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804"
Jul 27 21:03:56.519458 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.519437    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:03:56.523543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.523518    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:03:56.524773 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.524755    2552 scope.go:117] "RemoveContainer" containerID="042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7"
Jul 27 21:03:56.530972 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.530955    2552 scope.go:117] "RemoveContainer" containerID="6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c"
Jul 27 21:03:56.538606 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.538588    2552 scope.go:117] "RemoveContainer" containerID="f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80"
Jul 27 21:03:56.544905 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.544889    2552 scope.go:117] "RemoveContainer" containerID="168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478"
Jul 27 21:03:56.545974 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.545942    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478\": container with ID starting with 168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478 not found: ID does not exist" containerID="168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478"
Jul 27 21:03:56.546072 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.545985    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478"} err="failed to get container status \"168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478\": rpc error: code = NotFound desc = could not find container \"168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478\": container with ID starting with 168d00d1f449646bb46387665a0aa24f769de21e2f923db0c36c407121175478 not found: ID does not exist"
Jul 27 21:03:56.546072 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.546011    2552 scope.go:117] "RemoveContainer" containerID="d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c"
Jul 27 21:03:56.546342 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.546321    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c\": container with ID starting with d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c not found: ID does not exist" containerID="d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c"
Jul 27 21:03:56.546444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.546348    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c"} err="failed to get container status \"d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c\": rpc error: code = NotFound desc = could not find container \"d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c\": container with ID starting with d46ac64bdcb4e6a1ac25948e32e5cb278a4c2dd3e4f7228e544f5773c94b765c not found: ID does not exist"
Jul 27 21:03:56.546444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.546394    2552 scope.go:117] "RemoveContainer" containerID="ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88"
Jul 27 21:03:56.546773 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.546753    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88\": container with ID starting with ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88 not found: ID does not exist" containerID="ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88"
Jul 27 21:03:56.546863 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.546777    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88"} err="failed to get container status \"ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88\": rpc error: code = NotFound desc = could not find container \"ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88\": container with ID starting with ba40bb49114cc74a26ca050dc5b7061ec603b740f5c44af7a728df2c0936fc88 not found: ID does not exist"
Jul 27 21:03:56.546863 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.546796    2552 scope.go:117] "RemoveContainer" containerID="f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804"
Jul 27 21:03:56.547050 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.547030    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804\": container with ID starting with f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804 not found: ID does not exist" containerID="f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804"
Jul 27 21:03:56.547115 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547057    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804"} err="failed to get container status \"f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804\": rpc error: code = NotFound desc = could not find container \"f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804\": container with ID starting with f5a8e776089ce8a08d33b13b64f9b8465e7271c5cf2c8a4c8152f3f31a536804 not found: ID does not exist"
Jul 27 21:03:56.547115 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547077    2552 scope.go:117] "RemoveContainer" containerID="042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7"
Jul 27 21:03:56.547206 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547189    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:03:56.547319 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.547304    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7\": container with ID starting with 042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7 not found: ID does not exist" containerID="042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7"
Jul 27 21:03:56.547374 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547324    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7"} err="failed to get container status \"042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7\": rpc error: code = NotFound desc = could not find container \"042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7\": container with ID starting with 042ee45569e1bdd070899a85063ff449accd19f1c01cddffc56725d6cbd5a9e7 not found: ID does not exist"
Jul 27 21:03:56.547374 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547337    2552 scope.go:117] "RemoveContainer" containerID="6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c"
Jul 27 21:03:56.547522 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547506    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="prometheus"
Jul 27 21:03:56.547557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547525    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="prometheus"
Jul 27 21:03:56.547557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547540    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-thanos"
Jul 27 21:03:56.547557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547546    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-thanos"
Jul 27 21:03:56.547557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547556    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="thanos-sidecar"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547563    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="thanos-sidecar"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547573    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547578    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547590    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="init-config-reloader"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547596    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="init-config-reloader"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.547522    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c\": container with ID starting with 6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c not found: ID does not exist" containerID="6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547602    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-web"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547608    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-web"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547615    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="config-reloader"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547613    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c"} err="failed to get container status \"6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c\": rpc error: code = NotFound desc = could not find container \"6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c\": container with ID starting with 6eca916cd0252e854d1d110f1e797094ff0d49a37c641fd00329e3188eba500c not found: ID does not exist"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547632    2552 scope.go:117] "RemoveContainer" containerID="f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80"
Jul 27 21:03:56.547689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547620    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="config-reloader"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547726    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="prometheus"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547736    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="config-reloader"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547745    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-thanos"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547755    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="thanos-sidecar"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547765    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy-web"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547772    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" containerName="kube-rbac-proxy"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:03:56.547840    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80\": container with ID starting with f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80 not found: ID does not exist" containerID="f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80"
Jul 27 21:03:56.548067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.547857    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80"} err="failed to get container status \"f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80\": rpc error: code = NotFound desc = could not find container \"f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80\": container with ID starting with f9ea7f177168badb17e021f35ab8b30f95bae80c41fc52afe7c69d648dae8d80 not found: ID does not exist"
Jul 27 21:03:56.551414 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.551400    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.553928 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.553905    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 27 21:03:56.554063 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554045    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 27 21:03:56.554131 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554089    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 27 21:03:56.554131 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554122    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4kn4w\""
Jul 27 21:03:56.554265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554145    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 27 21:03:56.554265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554149    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 27 21:03:56.554547 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554533    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 27 21:03:56.555059 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554929    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-632repka64g4l\""
Jul 27 21:03:56.555059 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554954    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 27 21:03:56.555059 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.554965    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 27 21:03:56.555059 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.555019    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 27 21:03:56.555344 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.555326    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 27 21:03:56.555415 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.555389    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 27 21:03:56.555575 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.555559    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 27 21:03:56.559088 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.559071    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 27 21:03:56.560417 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.560400    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 27 21:03:56.565746 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.565727    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:03:56.656701 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656671    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656701 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656703    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656725    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656743    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656766    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656782    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656797    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656815    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc4a7d48-4806-401c-b331-1f1109371f35-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656854    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.656884 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656880    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656903    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656927    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d48-4806-401c-b331-1f1109371f35-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.656980    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657012    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657028    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657041    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trj2c\" (UniqueName: \"kubernetes.io/projected/dc4a7d48-4806-401c-b331-1f1109371f35-kube-api-access-trj2c\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657060    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657340 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657107    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657340 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657131    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.657340 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.657147    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758071 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.757997    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758071 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758038    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758071 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758055    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758092    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758125    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758150    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc4a7d48-4806-401c-b331-1f1109371f35-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758181    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758211    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758274    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758310    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d48-4806-401c-b331-1f1109371f35-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758338    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758374    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758396    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758418    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trj2c\" (UniqueName: \"kubernetes.io/projected/dc4a7d48-4806-401c-b331-1f1109371f35-kube-api-access-trj2c\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758449    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758485    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758512    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758535    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758563    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.758669 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758597    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.759324 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.759018    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.759324 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.758987    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.759324 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.759097    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.759748 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.759721    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.761557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.761527    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dc4a7d48-4806-401c-b331-1f1109371f35-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.761557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.761531    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.761709 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.761564    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.761709 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.761564    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.761709 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.761534    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dc4a7d48-4806-401c-b331-1f1109371f35-config-out\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.762417 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.762386    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.762529 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.762416    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.762620 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.762596    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.762714 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.762693    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.763167 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.763145    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.763690 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.763662    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-web-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.763825 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.763807    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-config\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.764077 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.764061    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.764510 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.764491    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dc4a7d48-4806-401c-b331-1f1109371f35-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.764703 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.764685    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dc4a7d48-4806-401c-b331-1f1109371f35-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.767372 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.767356    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trj2c\" (UniqueName: \"kubernetes.io/projected/dc4a7d48-4806-401c-b331-1f1109371f35-kube-api-access-trj2c\") pod \"prometheus-k8s-0\" (UID: \"dc4a7d48-4806-401c-b331-1f1109371f35\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.863591 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.863565    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:03:56.996999 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:56.996973    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 27 21:03:56.999017 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:03:56.998975    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc4a7d48_4806_401c_b331_1f1109371f35.slice/crio-c18bc93cdf950f740e6bfa0cbf8bd8f73170705d2a8f27dd382a7731ab81f25f WatchSource:0}: Error finding container c18bc93cdf950f740e6bfa0cbf8bd8f73170705d2a8f27dd382a7731ab81f25f: Status 404 returned error can't find the container with id c18bc93cdf950f740e6bfa0cbf8bd8f73170705d2a8f27dd382a7731ab81f25f
Jul 27 21:03:57.199057 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:57.199026    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13ca80bb-d95a-45be-b816-731b9f451e4c" path="/var/lib/kubelet/pods/13ca80bb-d95a-45be-b816-731b9f451e4c/volumes"
Jul 27 21:03:57.500576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:57.500536    2552 generic.go:356] "Generic (PLEG): container finished" podID="dc4a7d48-4806-401c-b331-1f1109371f35" containerID="0f1a25d144668bfb09c90461a1dfaa65b1f159f5cd0fd517e78749677ca867d3" exitCode=0
Jul 27 21:03:57.500754 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:57.500629    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerDied","Data":"0f1a25d144668bfb09c90461a1dfaa65b1f159f5cd0fd517e78749677ca867d3"}
Jul 27 21:03:57.500754 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:57.500703    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"c18bc93cdf950f740e6bfa0cbf8bd8f73170705d2a8f27dd382a7731ab81f25f"}
Jul 27 21:03:58.511515 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.511478    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"968145ff2df87a6f68550477890ec7a8c753bd9e1f681a7a240759cbe8eb0b43"}
Jul 27 21:03:58.511954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.511595    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"009e21d1a15ca3d3f68353fbc892448c5787eb455c9ed0416c6ba123e711da05"}
Jul 27 21:03:58.511954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.511640    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"83312537200eea3dddd170f3ae1985396d676dcec3c9170b6307f3f6f05b66f2"}
Jul 27 21:03:58.511954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.511679    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"338bc807c093efffdd6448f5c0b27e8f882c03b89aef817e558c1a7f40a28704"}
Jul 27 21:03:58.511954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.511714    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"ed07c857fe62c97cd92e218d294a790652351817f235c8a181af1ba25748eb6c"}
Jul 27 21:03:58.511954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.511768    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dc4a7d48-4806-401c-b331-1f1109371f35","Type":"ContainerStarted","Data":"00d33beb8f560b3a9289b12e12f893fb3ae0db1c9411f221f264f3298ca0ea51"}
Jul 27 21:03:58.539393 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:03:58.539331    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.520891333 podStartE2EDuration="2.539309215s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.417882ms" isStatefulPod=false podCreationTimestamp="2026-07-27 21:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:03:58.537635372 +0000 UTC m=+381.840880160" watchObservedRunningTime="2026-07-27 21:03:58.539309215 +0000 UTC m=+381.842553981"
Jul 27 21:04:01.864533 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:04:01.864495    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:04:56.864000 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:04:56.863968    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:04:56.879709 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:04:56.879682    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:04:57.735845 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:04:57.735820    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 27 21:07:21.751681 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.751652    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"]
Jul 27 21:07:21.758361 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.758333    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.760805 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.760776    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-g5q6d\""
Jul 27 21:07:21.760956 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.760932    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 27 21:07:21.761061 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.761032    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 27 21:07:21.761420 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.761395    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 27 21:07:21.761493 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.761401    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 27 21:07:21.772156 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.772138    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"]
Jul 27 21:07:21.872098 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.872067    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.872244 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.872125    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbvc8\" (UniqueName: \"kubernetes.io/projected/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-kube-api-access-rbvc8\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.872244 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.872153    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.872244 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.872178    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-tmp\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.872367 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.872291    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-manifests\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.973220 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.973195    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.973378 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.973255    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbvc8\" (UniqueName: \"kubernetes.io/projected/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-kube-api-access-rbvc8\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.973378 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.973279    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.973378 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.973303    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-tmp\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.973378 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.973332    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-manifests\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.973689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.973670    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-manifests\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.975583 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.975553    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-tmp\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.975816 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.975797    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.975891 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.975802    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:21.988961 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:21.988938    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbvc8\" (UniqueName: \"kubernetes.io/projected/ad9c960e-3417-46d6-a4ed-eda9659a0f5d-kube-api-access-rbvc8\") pod \"opendatahub-operator-controller-manager-798cd98846-6b6nw\" (UID: \"ad9c960e-3417-46d6-a4ed-eda9659a0f5d\") " pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:22.069117 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:22.069056    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:22.209808 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:22.209676    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"]
Jul 27 21:07:22.212299 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:07:22.212268    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad9c960e_3417_46d6_a4ed_eda9659a0f5d.slice/crio-81b96b732be3fc4853cde57964f01c748807cfdd5a5206aa866fe3e56e63875a WatchSource:0}: Error finding container 81b96b732be3fc4853cde57964f01c748807cfdd5a5206aa866fe3e56e63875a: Status 404 returned error can't find the container with id 81b96b732be3fc4853cde57964f01c748807cfdd5a5206aa866fe3e56e63875a
Jul 27 21:07:22.230978 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:22.230954    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw" event={"ID":"ad9c960e-3417-46d6-a4ed-eda9659a0f5d","Type":"ContainerStarted","Data":"81b96b732be3fc4853cde57964f01c748807cfdd5a5206aa866fe3e56e63875a"}
Jul 27 21:07:26.249441 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:26.249406    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw" event={"ID":"ad9c960e-3417-46d6-a4ed-eda9659a0f5d","Type":"ContainerStarted","Data":"6a3a1a9bd3c9c01cf78c61e8c3bde20e7ebb851c271c8ec0a36213d85b466235"}
Jul 27 21:07:27.254131 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:27.254099    2552 generic.go:356] "Generic (PLEG): container finished" podID="ad9c960e-3417-46d6-a4ed-eda9659a0f5d" containerID="6a3a1a9bd3c9c01cf78c61e8c3bde20e7ebb851c271c8ec0a36213d85b466235" exitCode=0
Jul 27 21:07:27.254650 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:27.254167    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw" event={"ID":"ad9c960e-3417-46d6-a4ed-eda9659a0f5d","Type":"ContainerDied","Data":"6a3a1a9bd3c9c01cf78c61e8c3bde20e7ebb851c271c8ec0a36213d85b466235"}
Jul 27 21:07:28.259067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:28.259036    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw" event={"ID":"ad9c960e-3417-46d6-a4ed-eda9659a0f5d","Type":"ContainerStarted","Data":"87116a39d84ef150a42be05de5255d032c6410f9d4d6d4652c47d148bcd01e41"}
Jul 27 21:07:28.259491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:28.259178    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:28.283050 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:28.283005    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw" podStartSLOduration=2.704543168 podStartE2EDuration="7.282988567s" totalImagesPullingTime="3.565526943s" totalInitContainerRuntime="1.012918456s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:07:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:07:28.282121931 +0000 UTC m=+591.585366707" watchObservedRunningTime="2026-07-27 21:07:28.282988567 +0000 UTC m=+591.586233338"
Jul 27 21:07:39.265062 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:39.265032    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-798cd98846-6b6nw"
Jul 27 21:07:40.287093 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.287055    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"]
Jul 27 21:07:40.290826 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.290804    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.293751 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.293731    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 27 21:07:40.294045 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.294028    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 27 21:07:40.294679 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.294659    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hzf8w\""
Jul 27 21:07:40.294773 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.294690    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 27 21:07:40.294962 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.294946    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 27 21:07:40.295015 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.294997    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 27 21:07:40.300139 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.300120    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"]
Jul 27 21:07:40.411265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.411213    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3bf83e4d-b229-43e3-b485-ffd6e175a928-manager-config\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.411423 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.411296    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3bf83e4d-b229-43e3-b485-ffd6e175a928-metrics-cert\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.411423 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.411342    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rx4p\" (UniqueName: \"kubernetes.io/projected/3bf83e4d-b229-43e3-b485-ffd6e175a928-kube-api-access-2rx4p\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.411423 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.411371    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf83e4d-b229-43e3-b485-ffd6e175a928-cert\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.511938 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.511900    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3bf83e4d-b229-43e3-b485-ffd6e175a928-manager-config\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.512094 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.511955    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3bf83e4d-b229-43e3-b485-ffd6e175a928-metrics-cert\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.512094 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.512000    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rx4p\" (UniqueName: \"kubernetes.io/projected/3bf83e4d-b229-43e3-b485-ffd6e175a928-kube-api-access-2rx4p\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.512094 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.512023    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf83e4d-b229-43e3-b485-ffd6e175a928-cert\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.512564 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.512545    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3bf83e4d-b229-43e3-b485-ffd6e175a928-manager-config\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.514558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.514538    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3bf83e4d-b229-43e3-b485-ffd6e175a928-metrics-cert\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.514662 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.514646    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf83e4d-b229-43e3-b485-ffd6e175a928-cert\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.520536 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.520503    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rx4p\" (UniqueName: \"kubernetes.io/projected/3bf83e4d-b229-43e3-b485-ffd6e175a928-kube-api-access-2rx4p\") pod \"lws-controller-manager-75c6cc9f98-7ct2k\" (UID: \"3bf83e4d-b229-43e3-b485-ffd6e175a928\") " pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.599888 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.599820    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:40.739777 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.739749    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"]
Jul 27 21:07:40.742563 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:07:40.742535    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf83e4d_b229_43e3_b485_ffd6e175a928.slice/crio-f4b505345045fd392eab0be6c6860af558046e3bb931c63928d561678887ce75 WatchSource:0}: Error finding container f4b505345045fd392eab0be6c6860af558046e3bb931c63928d561678887ce75: Status 404 returned error can't find the container with id f4b505345045fd392eab0be6c6860af558046e3bb931c63928d561678887ce75
Jul 27 21:07:40.744766 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:40.744748    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:07:41.307009 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:41.306971    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k" event={"ID":"3bf83e4d-b229-43e3-b485-ffd6e175a928","Type":"ContainerStarted","Data":"f4b505345045fd392eab0be6c6860af558046e3bb931c63928d561678887ce75"}
Jul 27 21:07:44.321118 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:44.321083    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k" event={"ID":"3bf83e4d-b229-43e3-b485-ffd6e175a928","Type":"ContainerStarted","Data":"4d68f860e0525209a87259181405142a9376d7383db727401b79a43a7fdcd605"}
Jul 27 21:07:44.321554 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:44.321205    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:07:44.338473 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:44.338422    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k" podStartSLOduration=1.679916178 podStartE2EDuration="4.338408389s" totalImagesPullingTime="2.658492211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:07:44.3370231 +0000 UTC m=+607.640267865" watchObservedRunningTime="2026-07-27 21:07:44.338408389 +0000 UTC m=+607.641653154"
Jul 27 21:07:46.529539 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.529495    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"]
Jul 27 21:07:46.531774 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.531758    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.535171 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.535146    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 27 21:07:46.535405 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.535286    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 27 21:07:46.535783 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.535762    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 27 21:07:46.536408 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.536378    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-25pjb\""
Jul 27 21:07:46.536509 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.536416    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 27 21:07:46.546584 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.546560    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"]
Jul 27 21:07:46.666135 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.666103    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-tls-certs\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.666135 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.666139    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mpw\" (UniqueName: \"kubernetes.io/projected/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-kube-api-access-j7mpw\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.666364 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.666158    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-tmp\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.767277 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.767245    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mpw\" (UniqueName: \"kubernetes.io/projected/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-kube-api-access-j7mpw\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.767277 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.767280    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-tmp\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.767488 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.767344    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-tls-certs\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.769650 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.769620    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-tmp\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.769867 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.769851    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-tls-certs\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.776261 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.776216    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mpw\" (UniqueName: \"kubernetes.io/projected/a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc-kube-api-access-j7mpw\") pod \"kube-auth-proxy-d89c5fb64-sz68b\" (UID: \"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc\") " pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.841377 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.841332    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"
Jul 27 21:07:46.964829 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:46.964796    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b"]
Jul 27 21:07:46.967763 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:07:46.967737    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d109f8_d8bf_4a7c_b31d_9c4606cbdfcc.slice/crio-eb92a544b78f4f936308d7cb05c88eb612cc437c37409b4a792f1f6dfc8bfa57 WatchSource:0}: Error finding container eb92a544b78f4f936308d7cb05c88eb612cc437c37409b4a792f1f6dfc8bfa57: Status 404 returned error can't find the container with id eb92a544b78f4f936308d7cb05c88eb612cc437c37409b4a792f1f6dfc8bfa57
Jul 27 21:07:47.333054 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:47.333020    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b" event={"ID":"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc","Type":"ContainerStarted","Data":"eb92a544b78f4f936308d7cb05c88eb612cc437c37409b4a792f1f6dfc8bfa57"}
Jul 27 21:07:50.347490 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:50.347449    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b" event={"ID":"a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc","Type":"ContainerStarted","Data":"826fecbbd10094e395d235862a0efeab231b1d7edb255c33d39938f66adcebb4"}
Jul 27 21:07:50.369876 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:50.369826    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d89c5fb64-sz68b" podStartSLOduration=1.911659124 podStartE2EDuration="4.369810128s" totalImagesPullingTime="2.458151004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:07:50.367705623 +0000 UTC m=+613.670950388" watchObservedRunningTime="2026-07-27 21:07:50.369810128 +0000 UTC m=+613.673054932"
Jul 27 21:07:55.327424 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:07:55.327397    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-75c6cc9f98-7ct2k"
Jul 27 21:09:25.880983 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.880953    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"]
Jul 27 21:09:25.883040 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.883024    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:25.885874 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.885851    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 27 21:09:25.886038 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.885850    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 27 21:09:25.886038 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.885961    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 27 21:09:25.886538 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.886515    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 27 21:09:25.886660 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.886569    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lljbj\""
Jul 27 21:09:25.897749 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.897732    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"]
Jul 27 21:09:25.945812 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.945781    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:25.945949 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.945839    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcbmr\" (UniqueName: \"kubernetes.io/projected/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-kube-api-access-zcbmr\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:25.945949 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:25.945871    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.046284 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.046249    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.046420 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.046318    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcbmr\" (UniqueName: \"kubernetes.io/projected/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-kube-api-access-zcbmr\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.046420 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.046345    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.046420 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:09:26.046412    2552 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 27 21:09:26.046531 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:09:26.046493    2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-plugin-serving-cert podName:ba7b0df9-c0f0-4a8e-b548-df60552a0b03 nodeName:}" failed. No retries permitted until 2026-07-27 21:09:26.546473272 +0000 UTC m=+709.849718018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-s4h2g" (UID: "ba7b0df9-c0f0-4a8e-b548-df60552a0b03") : secret "plugin-serving-cert" not found
Jul 27 21:09:26.046997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.046980    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.057689 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.057665    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcbmr\" (UniqueName: \"kubernetes.io/projected/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-kube-api-access-zcbmr\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.550047 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.550010    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.552557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.552535    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba7b0df9-c0f0-4a8e-b548-df60552a0b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s4h2g\" (UID: \"ba7b0df9-c0f0-4a8e-b548-df60552a0b03\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.792166 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.792134    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"
Jul 27 21:09:26.936154 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:26.936122    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g"]
Jul 27 21:09:26.939500 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:09:26.939469    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba7b0df9_c0f0_4a8e_b548_df60552a0b03.slice/crio-b4e915cdc106c970c818ceb085281edbe7f9786d2f2a3a64cde495d38a8ba2ea WatchSource:0}: Error finding container b4e915cdc106c970c818ceb085281edbe7f9786d2f2a3a64cde495d38a8ba2ea: Status 404 returned error can't find the container with id b4e915cdc106c970c818ceb085281edbe7f9786d2f2a3a64cde495d38a8ba2ea
Jul 27 21:09:27.708663 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:27.708630    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g" event={"ID":"ba7b0df9-c0f0-4a8e-b548-df60552a0b03","Type":"ContainerStarted","Data":"b4e915cdc106c970c818ceb085281edbe7f9786d2f2a3a64cde495d38a8ba2ea"}
Jul 27 21:09:38.467501 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.467467    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"]
Jul 27 21:09:38.475909 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.475882    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.478883 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.478860    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sjd55\""
Jul 27 21:09:38.487418 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.487393    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"]
Jul 27 21:09:38.558288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.558260    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f0358e9-578c-407f-99f7-c02aac748c64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x6sch\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.558288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.558293    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wp7l\" (UniqueName: \"kubernetes.io/projected/6f0358e9-578c-407f-99f7-c02aac748c64-kube-api-access-5wp7l\") pod \"kuadrant-operator-controller-manager-86878c769f-x6sch\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.659197 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.659152    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f0358e9-578c-407f-99f7-c02aac748c64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x6sch\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.659197 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.659195    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wp7l\" (UniqueName: \"kubernetes.io/projected/6f0358e9-578c-407f-99f7-c02aac748c64-kube-api-access-5wp7l\") pod \"kuadrant-operator-controller-manager-86878c769f-x6sch\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.659584 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.659559    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f0358e9-578c-407f-99f7-c02aac748c64-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x6sch\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.668472 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.668451    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wp7l\" (UniqueName: \"kubernetes.io/projected/6f0358e9-578c-407f-99f7-c02aac748c64-kube-api-access-5wp7l\") pod \"kuadrant-operator-controller-manager-86878c769f-x6sch\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:38.788316 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:38.788201    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:51.507886 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:51.507862    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"]
Jul 27 21:09:51.510562 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:09:51.510529    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0358e9_578c_407f_99f7_c02aac748c64.slice/crio-6be81c8f3a1dd66ac6d5c593b6a1af387810ddffbcf9f000d147167c0f95fe41 WatchSource:0}: Error finding container 6be81c8f3a1dd66ac6d5c593b6a1af387810ddffbcf9f000d147167c0f95fe41: Status 404 returned error can't find the container with id 6be81c8f3a1dd66ac6d5c593b6a1af387810ddffbcf9f000d147167c0f95fe41
Jul 27 21:09:51.822293 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:51.822265    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch" event={"ID":"6f0358e9-578c-407f-99f7-c02aac748c64","Type":"ContainerStarted","Data":"6be81c8f3a1dd66ac6d5c593b6a1af387810ddffbcf9f000d147167c0f95fe41"}
Jul 27 21:09:52.827354 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:52.827323    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g" event={"ID":"ba7b0df9-c0f0-4a8e-b548-df60552a0b03","Type":"ContainerStarted","Data":"04acf29e44d5191fb2caa839dd0bea5f26857f4f512ea8856ceeb92e4b51725a"}
Jul 27 21:09:52.844309 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:52.844265    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s4h2g" podStartSLOduration=3.017684937 podStartE2EDuration="27.844250194s" totalImagesPullingTime="24.826565257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:09:52.843343921 +0000 UTC m=+736.146588687" watchObservedRunningTime="2026-07-27 21:09:52.844250194 +0000 UTC m=+736.147494983"
Jul 27 21:09:57.852387 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:57.852355    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch" event={"ID":"6f0358e9-578c-407f-99f7-c02aac748c64","Type":"ContainerStarted","Data":"cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10"}
Jul 27 21:09:57.852813 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:57.852456    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:09:57.872499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:09:57.872446    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch" podStartSLOduration=14.181488538 podStartE2EDuration="19.872432045s" totalImagesPullingTime="5.690943507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:09:57.870308278 +0000 UTC m=+741.173553044" watchObservedRunningTime="2026-07-27 21:09:57.872432045 +0000 UTC m=+741.175676811"
Jul 27 21:10:08.859181 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:10:08.859152    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:12:10.999178 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:10.999147    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"]
Jul 27 21:12:11.001722 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.001707    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.004149 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.004123    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 27 21:12:11.005150 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.005129    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b6v8k\""
Jul 27 21:12:11.005307 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.005142    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 27 21:12:11.005307 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.005159    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 27 21:12:11.011060 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.011038    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"]
Jul 27 21:12:11.108357 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.108319    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3e06410-7430-4a13-a8ac-271a60db6a68-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.108543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.108363    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.108543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.108394    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.108543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.108418    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bhgn\" (UniqueName: \"kubernetes.io/projected/f3e06410-7430-4a13-a8ac-271a60db6a68-kube-api-access-7bhgn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.108543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.108471    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.108543 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.108518    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209024 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.208997    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209167 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209029    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bhgn\" (UniqueName: \"kubernetes.io/projected/f3e06410-7430-4a13-a8ac-271a60db6a68-kube-api-access-7bhgn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209208 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209166    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209246    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209313 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209302    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3e06410-7430-4a13-a8ac-271a60db6a68-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209350 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209328    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209405 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209389    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209600 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209581    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.209684 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.209656    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.211620 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.211601    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3e06410-7430-4a13-a8ac-271a60db6a68-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.211923 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.211903    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3e06410-7430-4a13-a8ac-271a60db6a68-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.217272 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.217251    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bhgn\" (UniqueName: \"kubernetes.io/projected/f3e06410-7430-4a13-a8ac-271a60db6a68-kube-api-access-7bhgn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq\" (UID: \"f3e06410-7430-4a13-a8ac-271a60db6a68\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.312435 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.312370    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:11.443199 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:11.443176    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"]
Jul 27 21:12:11.445305 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:12:11.445278    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e06410_7430_4a13_a8ac_271a60db6a68.slice/crio-3aa6e4056ef6ed2df388e9ead9b0dc15273c948734138adcf3fd9d773897c027 WatchSource:0}: Error finding container 3aa6e4056ef6ed2df388e9ead9b0dc15273c948734138adcf3fd9d773897c027: Status 404 returned error can't find the container with id 3aa6e4056ef6ed2df388e9ead9b0dc15273c948734138adcf3fd9d773897c027
Jul 27 21:12:12.412993 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:12.412958    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq" event={"ID":"f3e06410-7430-4a13-a8ac-271a60db6a68","Type":"ContainerStarted","Data":"3aa6e4056ef6ed2df388e9ead9b0dc15273c948734138adcf3fd9d773897c027"}
Jul 27 21:12:13.422444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:13.422404    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq" event={"ID":"f3e06410-7430-4a13-a8ac-271a60db6a68","Type":"ContainerStarted","Data":"877e6e2caf648d9f6df3c54752fe65aa59dcbbc7e838d8b65f8c6864066ccbcc"}
Jul 27 21:12:13.438982 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:13.438935    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq" podStartSLOduration=2.323608794 podStartE2EDuration="3.438919577s" totalImagesPullingTime="1.115310783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:12:13.438338863 +0000 UTC m=+876.741583628" watchObservedRunningTime="2026-07-27 21:12:13.438919577 +0000 UTC m=+876.742164341"
Jul 27 21:12:21.313418 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:21.313384    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:21.313900 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:21.313470    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:21.325676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:21.325652    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:21.467652 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:21.467627    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq"
Jul 27 21:12:54.791167 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.791129    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"]
Jul 27 21:12:54.793595 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.793576    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.796119 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.796100    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 27 21:12:54.803461 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.803435    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"]
Jul 27 21:12:54.871488 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.871459    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db6ns\" (UniqueName: \"kubernetes.io/projected/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-kube-api-access-db6ns\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.871668 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.871497    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.871668 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.871513    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.871668 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.871632    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.871668 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.871664    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.871862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.871742    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.972748 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.972714    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db6ns\" (UniqueName: \"kubernetes.io/projected/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-kube-api-access-db6ns\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.972914 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.972758    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.972914 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.972782    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.972914 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.972822    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.972914 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.972846    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.972914 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.972883    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.973167 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.973146    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.973204 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.973191    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.973340 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.973323    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.975212 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.975195    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.975483 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.975465    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:54.980852 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:54.980823    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db6ns\" (UniqueName: \"kubernetes.io/projected/b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2-kube-api-access-db6ns\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5\" (UID: \"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:55.104125 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:55.104035    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:12:55.232084 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:55.232060    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"]
Jul 27 21:12:55.234361 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:12:55.234332    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb951cf7c_f233_4ab7_b0c6_905d6aa1d3a2.slice/crio-b3c1e0eed2f567a8a51fcb8018d99a82ce861d0e85486c62680a0c9f9cc82e7b WatchSource:0}: Error finding container b3c1e0eed2f567a8a51fcb8018d99a82ce861d0e85486c62680a0c9f9cc82e7b: Status 404 returned error can't find the container with id b3c1e0eed2f567a8a51fcb8018d99a82ce861d0e85486c62680a0c9f9cc82e7b
Jul 27 21:12:55.236058 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:55.236041    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:12:55.597746 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:55.597705    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5" event={"ID":"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2","Type":"ContainerStarted","Data":"f2873ff36cc635abd77562b16aecf1a26af3d52538ecd667ba39c9cc052c544f"}
Jul 27 21:12:55.597903 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:55.597765    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5" event={"ID":"b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2","Type":"ContainerStarted","Data":"b3c1e0eed2f567a8a51fcb8018d99a82ce861d0e85486c62680a0c9f9cc82e7b"}
Jul 27 21:12:55.616398 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:12:55.616346    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5" podStartSLOduration=1.430251125 podStartE2EDuration="1.616330673s" totalImagesPullingTime="186.079548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:12:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:12:55.61449329 +0000 UTC m=+918.917738054" watchObservedRunningTime="2026-07-27 21:12:55.616330673 +0000 UTC m=+918.919575438"
Jul 27 21:13:05.104550 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:05.104517    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:13:05.104946 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:05.104588    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:13:05.116830 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:05.116811    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:13:05.655706 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:05.655679    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5"
Jul 27 21:13:15.540182 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.540150    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"]
Jul 27 21:13:15.542647 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.542632    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.544982 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.544965    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 27 21:13:15.585766 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.585742    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"]
Jul 27 21:13:15.647431 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.647407    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.647542 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.647437    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.647542 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.647466    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.647542 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.647486    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7f4f727-d404-4827-afed-8ffcc427cff5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.647648 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.647557    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.647648 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.647591    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6v27\" (UniqueName: \"kubernetes.io/projected/e7f4f727-d404-4827-afed-8ffcc427cff5-kube-api-access-f6v27\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748068 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748036    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748068 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748069    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6v27\" (UniqueName: \"kubernetes.io/projected/e7f4f727-d404-4827-afed-8ffcc427cff5-kube-api-access-f6v27\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748162    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748194    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748253    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748280    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7f4f727-d404-4827-afed-8ffcc427cff5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748541    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748597 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748574    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.748654 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.748634    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.750749 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.750723    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7f4f727-d404-4827-afed-8ffcc427cff5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.750940 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.750926    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7f4f727-d404-4827-afed-8ffcc427cff5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.789281 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.789254    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6v27\" (UniqueName: \"kubernetes.io/projected/e7f4f727-d404-4827-afed-8ffcc427cff5-kube-api-access-f6v27\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-69vxc\" (UID: \"e7f4f727-d404-4827-afed-8ffcc427cff5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.852841 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.852774    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:15.980820 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:15.980767    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"]
Jul 27 21:13:15.982955 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:13:15.982927    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f4f727_d404_4827_afed_8ffcc427cff5.slice/crio-187d539b0e93f6827f8d337d26846ce33135ae40d6d9e84ef6ab9b3d330891ca WatchSource:0}: Error finding container 187d539b0e93f6827f8d337d26846ce33135ae40d6d9e84ef6ab9b3d330891ca: Status 404 returned error can't find the container with id 187d539b0e93f6827f8d337d26846ce33135ae40d6d9e84ef6ab9b3d330891ca
Jul 27 21:13:16.693526 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:16.693490    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc" event={"ID":"e7f4f727-d404-4827-afed-8ffcc427cff5","Type":"ContainerStarted","Data":"f40fa34bde353b1ef0cbd02c2859d6ac249e9346588e9e4110b8a8b283dce07a"}
Jul 27 21:13:16.694008 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:16.693540    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc" event={"ID":"e7f4f727-d404-4827-afed-8ffcc427cff5","Type":"ContainerStarted","Data":"187d539b0e93f6827f8d337d26846ce33135ae40d6d9e84ef6ab9b3d330891ca"}
Jul 27 21:13:16.719028 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:16.718983    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc" podStartSLOduration=1.542342998 podStartE2EDuration="1.718967573s" totalImagesPullingTime="176.624575ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:13:16.717833311 +0000 UTC m=+940.021078075" watchObservedRunningTime="2026-07-27 21:13:16.718967573 +0000 UTC m=+940.022212339"
Jul 27 21:13:25.853870 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:25.853817    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:25.854298 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:25.853927    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:25.866568 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:25.866544    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:13:26.750545 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:13:26.750515    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-69vxc"
Jul 27 21:21:08.822868 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:08.822829    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"]
Jul 27 21:21:08.824521 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:08.823111    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch" podUID="6f0358e9-578c-407f-99f7-c02aac748c64" containerName="manager" containerID="cri-o://cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10" gracePeriod=10
Jul 27 21:21:11.165596 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.165573    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:21:11.311539 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.311459    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f0358e9-578c-407f-99f7-c02aac748c64-extensions-socket-volume\") pod \"6f0358e9-578c-407f-99f7-c02aac748c64\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") "
Jul 27 21:21:11.311539 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.311491    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wp7l\" (UniqueName: \"kubernetes.io/projected/6f0358e9-578c-407f-99f7-c02aac748c64-kube-api-access-5wp7l\") pod \"6f0358e9-578c-407f-99f7-c02aac748c64\" (UID: \"6f0358e9-578c-407f-99f7-c02aac748c64\") "
Jul 27 21:21:11.311820 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.311797    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f0358e9-578c-407f-99f7-c02aac748c64-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6f0358e9-578c-407f-99f7-c02aac748c64" (UID: "6f0358e9-578c-407f-99f7-c02aac748c64"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:21:11.313851 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.313826    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f0358e9-578c-407f-99f7-c02aac748c64-kube-api-access-5wp7l" (OuterVolumeSpecName: "kube-api-access-5wp7l") pod "6f0358e9-578c-407f-99f7-c02aac748c64" (UID: "6f0358e9-578c-407f-99f7-c02aac748c64"). InnerVolumeSpecName "kube-api-access-5wp7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:21:11.412732 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.412702    2552 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f0358e9-578c-407f-99f7-c02aac748c64-extensions-socket-volume\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:21:11.412732 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.412728    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wp7l\" (UniqueName: \"kubernetes.io/projected/6f0358e9-578c-407f-99f7-c02aac748c64-kube-api-access-5wp7l\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:21:11.717160 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.717130    2552 generic.go:356] "Generic (PLEG): container finished" podID="6f0358e9-578c-407f-99f7-c02aac748c64" containerID="cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10" exitCode=0
Jul 27 21:21:11.717344 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.717185    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"
Jul 27 21:21:11.717344 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.717213    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch" event={"ID":"6f0358e9-578c-407f-99f7-c02aac748c64","Type":"ContainerDied","Data":"cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10"}
Jul 27 21:21:11.717344 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.717297    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch" event={"ID":"6f0358e9-578c-407f-99f7-c02aac748c64","Type":"ContainerDied","Data":"6be81c8f3a1dd66ac6d5c593b6a1af387810ddffbcf9f000d147167c0f95fe41"}
Jul 27 21:21:11.717344 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.717329    2552 scope.go:117] "RemoveContainer" containerID="cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10"
Jul 27 21:21:11.727189 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.727175    2552 scope.go:117] "RemoveContainer" containerID="cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10"
Jul 27 21:21:11.727468 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:21:11.727449    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10\": container with ID starting with cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10 not found: ID does not exist" containerID="cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10"
Jul 27 21:21:11.727528 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.727478    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10"} err="failed to get container status \"cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10\": rpc error: code = NotFound desc = could not find container \"cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10\": container with ID starting with cb8e7ce1b6ecad677fd83dc2a7f9f4e2c7f07bc23758606f5ba5f702f1cc3e10 not found: ID does not exist"
Jul 27 21:21:11.737765 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.737744    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"]
Jul 27 21:21:11.742482 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:11.742465    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x6sch"]
Jul 27 21:21:13.197626 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:21:13.197593    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f0358e9-578c-407f-99f7-c02aac748c64" path="/var/lib/kubelet/pods/6f0358e9-578c-407f-99f7-c02aac748c64/volumes"
Jul 27 21:22:10.020864 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.020829    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"]
Jul 27 21:22:10.021384 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.021147    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f0358e9-578c-407f-99f7-c02aac748c64" containerName="manager"
Jul 27 21:22:10.021384 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.021157    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f0358e9-578c-407f-99f7-c02aac748c64" containerName="manager"
Jul 27 21:22:10.021384 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.021240    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f0358e9-578c-407f-99f7-c02aac748c64" containerName="manager"
Jul 27 21:22:10.023941 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.023926    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.026937 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.026916    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sjd55\""
Jul 27 21:22:10.040554 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.040531    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"]
Jul 27 21:22:10.095917 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.095884    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3472236-38ab-4c86-b69d-df139f36fee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzcqd\" (UID: \"f3472236-38ab-4c86-b69d-df139f36fee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.096035 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.095933    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxwl\" (UniqueName: \"kubernetes.io/projected/f3472236-38ab-4c86-b69d-df139f36fee4-kube-api-access-jmxwl\") pod \"kuadrant-operator-controller-manager-86878c769f-qzcqd\" (UID: \"f3472236-38ab-4c86-b69d-df139f36fee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.196338 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.196308    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3472236-38ab-4c86-b69d-df139f36fee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzcqd\" (UID: \"f3472236-38ab-4c86-b69d-df139f36fee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.196485 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.196351    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxwl\" (UniqueName: \"kubernetes.io/projected/f3472236-38ab-4c86-b69d-df139f36fee4-kube-api-access-jmxwl\") pod \"kuadrant-operator-controller-manager-86878c769f-qzcqd\" (UID: \"f3472236-38ab-4c86-b69d-df139f36fee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.196684 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.196665    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f3472236-38ab-4c86-b69d-df139f36fee4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qzcqd\" (UID: \"f3472236-38ab-4c86-b69d-df139f36fee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.204796 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.204778    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxwl\" (UniqueName: \"kubernetes.io/projected/f3472236-38ab-4c86-b69d-df139f36fee4-kube-api-access-jmxwl\") pod \"kuadrant-operator-controller-manager-86878c769f-qzcqd\" (UID: \"f3472236-38ab-4c86-b69d-df139f36fee4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.333046 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.332988    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.468077 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.468045    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"]
Jul 27 21:22:10.471722 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:22:10.471695    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3472236_38ab_4c86_b69d_df139f36fee4.slice/crio-a29bec6b9acd70bdef277428db16f6cd0727362e7fbb4e301d35f3dfcef1ced3 WatchSource:0}: Error finding container a29bec6b9acd70bdef277428db16f6cd0727362e7fbb4e301d35f3dfcef1ced3: Status 404 returned error can't find the container with id a29bec6b9acd70bdef277428db16f6cd0727362e7fbb4e301d35f3dfcef1ced3
Jul 27 21:22:10.964723 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.964686    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd" event={"ID":"f3472236-38ab-4c86-b69d-df139f36fee4","Type":"ContainerStarted","Data":"85c49fba48e0216175e416f6a8554b6f609c451c34c69b519e01b5abd9efc30e"}
Jul 27 21:22:10.964904 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.964738    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd" event={"ID":"f3472236-38ab-4c86-b69d-df139f36fee4","Type":"ContainerStarted","Data":"a29bec6b9acd70bdef277428db16f6cd0727362e7fbb4e301d35f3dfcef1ced3"}
Jul 27 21:22:10.964904 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.964765    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:22:10.985373 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:10.985332    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd" podStartSLOduration=1.985314234 podStartE2EDuration="1.985314234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:22:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:22:10.984452611 +0000 UTC m=+1474.287697375" watchObservedRunningTime="2026-07-27 21:22:10.985314234 +0000 UTC m=+1474.288558999"
Jul 27 21:22:21.978042 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:22:21.978016    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qzcqd"
Jul 27 21:38:14.688768 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.688680    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"]
Jul 27 21:38:14.692351 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.692327    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.694763 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.694740    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-16c965\"/\"kube-root-ca.crt\""
Jul 27 21:38:14.694896 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.694770    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-16c965\"/\"openshift-service-ca.crt\""
Jul 27 21:38:14.695611 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.695594    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-16c965\"/\"default-dockercfg-9fsh6\""
Jul 27 21:38:14.695971 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.695951    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-16c965\"/\"test-model-16c965-kserve-self-signed-certs\""
Jul 27 21:38:14.701189 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.701170    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"]
Jul 27 21:38:14.730971 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.730947    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-home\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.731101 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.730977    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-dshm\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.731101 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.730994    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-tmp-dir\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.731101 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.731014    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-model-cache\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.731210 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.731096    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbcbj\" (UniqueName: \"kubernetes.io/projected/23a41a87-9a04-46ea-a1d2-59d07415ae20-kube-api-access-fbcbj\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.731210 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.731140    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a41a87-9a04-46ea-a1d2-59d07415ae20-tls-certs\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.832581 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.832554    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-home\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.832581 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.832584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-dshm\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.832819 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.832601    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-tmp-dir\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.832819 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.832622    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-model-cache\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.832819 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.832655    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbcbj\" (UniqueName: \"kubernetes.io/projected/23a41a87-9a04-46ea-a1d2-59d07415ae20-kube-api-access-fbcbj\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.832819 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.832675    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a41a87-9a04-46ea-a1d2-59d07415ae20-tls-certs\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.833045 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.833004    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-home\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.833092 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.833013    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-tmp-dir\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.833126 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.833091    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-model-cache\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.835109 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.835090    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-dshm\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.835394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.835378    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a41a87-9a04-46ea-a1d2-59d07415ae20-tls-certs\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:14.840864 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:14.840842    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbcbj\" (UniqueName: \"kubernetes.io/projected/23a41a87-9a04-46ea-a1d2-59d07415ae20-kube-api-access-fbcbj\") pod \"test-model-16c965-kserve-55bd6b88f-vwkxw\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") " pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:15.003254 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:15.003150    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:15.134913 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:15.134885    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"]
Jul 27 21:38:15.138641 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:38:15.138618    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23a41a87_9a04_46ea_a1d2_59d07415ae20.slice/crio-d3f25559370e8b0c5bb77c4372c4d564c7306b46f9b9096468517f254cb14db8 WatchSource:0}: Error finding container d3f25559370e8b0c5bb77c4372c4d564c7306b46f9b9096468517f254cb14db8: Status 404 returned error can't find the container with id d3f25559370e8b0c5bb77c4372c4d564c7306b46f9b9096468517f254cb14db8
Jul 27 21:38:16.071931 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:16.071889    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw" event={"ID":"23a41a87-9a04-46ea-a1d2-59d07415ae20","Type":"ContainerStarted","Data":"8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb"}
Jul 27 21:38:16.071931 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:16.071935    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw" event={"ID":"23a41a87-9a04-46ea-a1d2-59d07415ae20","Type":"ContainerStarted","Data":"d3f25559370e8b0c5bb77c4372c4d564c7306b46f9b9096468517f254cb14db8"}
Jul 27 21:38:16.089037 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:16.088990    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw" podStartSLOduration=2.088974801 podStartE2EDuration="2.088974801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:38:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:38:16.087095272 +0000 UTC m=+2439.390340040" watchObservedRunningTime="2026-07-27 21:38:16.088974801 +0000 UTC m=+2439.392219567"
Jul 27 21:38:25.003677 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:25.003637    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:25.004162 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:25.003718    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:25.016571 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:25.016547    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:38:25.124724 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:38:25.124693    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:39:31.420162 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:31.420072    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"]
Jul 27 21:39:46.439512 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.439455    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw" podUID="23a41a87-9a04-46ea-a1d2-59d07415ae20" containerName="main" containerID="cri-o://8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb" gracePeriod=45
Jul 27 21:39:46.674001 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.673977    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:39:46.710203 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710143    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-home\") pod \"23a41a87-9a04-46ea-a1d2-59d07415ae20\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") "
Jul 27 21:39:46.710203 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710171    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a41a87-9a04-46ea-a1d2-59d07415ae20-tls-certs\") pod \"23a41a87-9a04-46ea-a1d2-59d07415ae20\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") "
Jul 27 21:39:46.710429 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710244    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-dshm\") pod \"23a41a87-9a04-46ea-a1d2-59d07415ae20\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") "
Jul 27 21:39:46.710429 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710271    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-model-cache\") pod \"23a41a87-9a04-46ea-a1d2-59d07415ae20\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") "
Jul 27 21:39:46.710429 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710297    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbcbj\" (UniqueName: \"kubernetes.io/projected/23a41a87-9a04-46ea-a1d2-59d07415ae20-kube-api-access-fbcbj\") pod \"23a41a87-9a04-46ea-a1d2-59d07415ae20\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") "
Jul 27 21:39:46.710429 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710337    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-tmp-dir\") pod \"23a41a87-9a04-46ea-a1d2-59d07415ae20\" (UID: \"23a41a87-9a04-46ea-a1d2-59d07415ae20\") "
Jul 27 21:39:46.710429 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710410    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-home" (OuterVolumeSpecName: "home") pod "23a41a87-9a04-46ea-a1d2-59d07415ae20" (UID: "23a41a87-9a04-46ea-a1d2-59d07415ae20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:39:46.710740 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710526    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-model-cache" (OuterVolumeSpecName: "model-cache") pod "23a41a87-9a04-46ea-a1d2-59d07415ae20" (UID: "23a41a87-9a04-46ea-a1d2-59d07415ae20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:39:46.710740 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710659    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-model-cache\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:39:46.710740 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710676    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-home\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:39:46.710740 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.710679    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "23a41a87-9a04-46ea-a1d2-59d07415ae20" (UID: "23a41a87-9a04-46ea-a1d2-59d07415ae20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:39:46.712619 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.712588    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a41a87-9a04-46ea-a1d2-59d07415ae20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "23a41a87-9a04-46ea-a1d2-59d07415ae20" (UID: "23a41a87-9a04-46ea-a1d2-59d07415ae20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:39:46.712732 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.712677    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-dshm" (OuterVolumeSpecName: "dshm") pod "23a41a87-9a04-46ea-a1d2-59d07415ae20" (UID: "23a41a87-9a04-46ea-a1d2-59d07415ae20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:39:46.712773 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.712726    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a41a87-9a04-46ea-a1d2-59d07415ae20-kube-api-access-fbcbj" (OuterVolumeSpecName: "kube-api-access-fbcbj") pod "23a41a87-9a04-46ea-a1d2-59d07415ae20" (UID: "23a41a87-9a04-46ea-a1d2-59d07415ae20"). InnerVolumeSpecName "kube-api-access-fbcbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:39:46.811650 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.811623    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/23a41a87-9a04-46ea-a1d2-59d07415ae20-tls-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:39:46.811650 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.811646    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-dshm\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:39:46.811650 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.811656    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbcbj\" (UniqueName: \"kubernetes.io/projected/23a41a87-9a04-46ea-a1d2-59d07415ae20-kube-api-access-fbcbj\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:39:46.811845 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:46.811665    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23a41a87-9a04-46ea-a1d2-59d07415ae20-tmp-dir\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:39:47.460576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.460483    2552 generic.go:356] "Generic (PLEG): container finished" podID="23a41a87-9a04-46ea-a1d2-59d07415ae20" containerID="8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb" exitCode=0
Jul 27 21:39:47.460576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.460531    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"
Jul 27 21:39:47.461047 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.460530    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw" event={"ID":"23a41a87-9a04-46ea-a1d2-59d07415ae20","Type":"ContainerDied","Data":"8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb"}
Jul 27 21:39:47.461047 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.460632    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw" event={"ID":"23a41a87-9a04-46ea-a1d2-59d07415ae20","Type":"ContainerDied","Data":"d3f25559370e8b0c5bb77c4372c4d564c7306b46f9b9096468517f254cb14db8"}
Jul 27 21:39:47.461047 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.460655    2552 scope.go:117] "RemoveContainer" containerID="8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb"
Jul 27 21:39:47.469097 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.469074    2552 scope.go:117] "RemoveContainer" containerID="8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb"
Jul 27 21:39:47.469349 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:39:47.469329    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb\": container with ID starting with 8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb not found: ID does not exist" containerID="8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb"
Jul 27 21:39:47.469442 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.469358    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb"} err="failed to get container status \"8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb\": rpc error: code = NotFound desc = could not find container \"8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb\": container with ID starting with 8568d07a99255fa536f5f983d512fc0fdadb0de16e7113d7eafa1722971c1bbb not found: ID does not exist"
Jul 27 21:39:47.476554 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.476527    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"]
Jul 27 21:39:47.481132 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:47.481111    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-16c965/test-model-16c965-kserve-55bd6b88f-vwkxw"]
Jul 27 21:39:49.196958 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:39:49.196921    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23a41a87-9a04-46ea-a1d2-59d07415ae20" path="/var/lib/kubelet/pods/23a41a87-9a04-46ea-a1d2-59d07415ae20/volumes"
Jul 27 21:40:32.463742 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.463702    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"]
Jul 27 21:40:32.464251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.464069    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23a41a87-9a04-46ea-a1d2-59d07415ae20" containerName="main"
Jul 27 21:40:32.464251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.464081    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="23a41a87-9a04-46ea-a1d2-59d07415ae20" containerName="main"
Jul 27 21:40:32.464251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.464151    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="23a41a87-9a04-46ea-a1d2-59d07415ae20" containerName="main"
Jul 27 21:40:32.467315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.467298    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:32.469562 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.469533    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22s5m\""
Jul 27 21:40:32.469767 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.469755    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 21:40:32.472551 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.472497    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"]
Jul 27 21:40:32.502818 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.502793    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9d0201-fd81-41e2-aa50-0244079c506b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf\" (UID: \"3f9d0201-fd81-41e2-aa50-0244079c506b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:32.603433 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.603398    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9d0201-fd81-41e2-aa50-0244079c506b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf\" (UID: \"3f9d0201-fd81-41e2-aa50-0244079c506b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:32.604029 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.604005    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9d0201-fd81-41e2-aa50-0244079c506b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf\" (UID: \"3f9d0201-fd81-41e2-aa50-0244079c506b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:32.779777 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.779700    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:32.913740 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.913714    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"]
Jul 27 21:40:32.916237 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:40:32.916196    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f9d0201_fd81_41e2_aa50_0244079c506b.slice/crio-3d6d6326ffff45864588faa086b5cffa693ebb8247bb5ed340777f7b062582db WatchSource:0}: Error finding container 3d6d6326ffff45864588faa086b5cffa693ebb8247bb5ed340777f7b062582db: Status 404 returned error can't find the container with id 3d6d6326ffff45864588faa086b5cffa693ebb8247bb5ed340777f7b062582db
Jul 27 21:40:32.918065 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:32.918043    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:40:33.666372 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:33.666340    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf" event={"ID":"3f9d0201-fd81-41e2-aa50-0244079c506b","Type":"ContainerStarted","Data":"3d6d6326ffff45864588faa086b5cffa693ebb8247bb5ed340777f7b062582db"}
Jul 27 21:40:36.682035 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:36.682000    2552 generic.go:356] "Generic (PLEG): container finished" podID="3f9d0201-fd81-41e2-aa50-0244079c506b" containerID="83efb27f78b21959d192d42b0f1836b80c21a0cfa83abeeba2fe28a2d3864022" exitCode=0
Jul 27 21:40:36.682491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:36.682069    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf" event={"ID":"3f9d0201-fd81-41e2-aa50-0244079c506b","Type":"ContainerDied","Data":"83efb27f78b21959d192d42b0f1836b80c21a0cfa83abeeba2fe28a2d3864022"}
Jul 27 21:40:37.813002 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:37.812973    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:37.850434 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:37.850407    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9d0201-fd81-41e2-aa50-0244079c506b-maas-api-service-ca\") pod \"3f9d0201-fd81-41e2-aa50-0244079c506b\" (UID: \"3f9d0201-fd81-41e2-aa50-0244079c506b\") "
Jul 27 21:40:37.850736 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:37.850716    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f9d0201-fd81-41e2-aa50-0244079c506b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f9d0201-fd81-41e2-aa50-0244079c506b" (UID: "3f9d0201-fd81-41e2-aa50-0244079c506b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:40:37.951138 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:37.951077    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f9d0201-fd81-41e2-aa50-0244079c506b-maas-api-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:40:38.692972 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:38.692935    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf" event={"ID":"3f9d0201-fd81-41e2-aa50-0244079c506b","Type":"ContainerDied","Data":"3d6d6326ffff45864588faa086b5cffa693ebb8247bb5ed340777f7b062582db"}
Jul 27 21:40:38.692972 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:38.692962    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"
Jul 27 21:40:38.693167 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:38.692991    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d6d6326ffff45864588faa086b5cffa693ebb8247bb5ed340777f7b062582db"
Jul 27 21:40:48.477468 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:48.477433    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"]
Jul 27 21:40:48.479514 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:48.479490    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-aba24e-qxlbf"]
Jul 27 21:40:49.199341 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:40:49.199311    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f9d0201-fd81-41e2-aa50-0244079c506b" path="/var/lib/kubelet/pods/3f9d0201-fd81-41e2-aa50-0244079c506b/volumes"
Jul 27 21:41:45.598585 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.598548    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"]
Jul 27 21:41:45.599098 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.598886    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f9d0201-fd81-41e2-aa50-0244079c506b" containerName="revoke-keys"
Jul 27 21:41:45.599098 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.598899    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f9d0201-fd81-41e2-aa50-0244079c506b" containerName="revoke-keys"
Jul 27 21:41:45.599098 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.598967    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f9d0201-fd81-41e2-aa50-0244079c506b" containerName="revoke-keys"
Jul 27 21:41:45.601700 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.601680    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.604100 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.604079    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"openshift-service-ca.crt\""
Jul 27 21:41:45.605096 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.605076    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"auth-test-model-3298cd-kserve-self-signed-certs\""
Jul 27 21:41:45.605179 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.605129    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"kube-root-ca.crt\""
Jul 27 21:41:45.605258 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.605151    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"default-dockercfg-sfxzx\""
Jul 27 21:41:45.611811 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.611791    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"]
Jul 27 21:41:45.725263 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.725214    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/165d32fb-e118-4c24-b257-79af1a95f629-tls-certs\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.725396 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.725272    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mhnp\" (UniqueName: \"kubernetes.io/projected/165d32fb-e118-4c24-b257-79af1a95f629-kube-api-access-9mhnp\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.725396 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.725330    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-dshm\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.725396 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.725363    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-tmp-dir\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.725498 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.725399    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-home\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.725498 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.725422    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-model-cache\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826185 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826152    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-home\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826356 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826190    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-model-cache\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826424 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826373    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/165d32fb-e118-4c24-b257-79af1a95f629-tls-certs\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826492 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826424    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mhnp\" (UniqueName: \"kubernetes.io/projected/165d32fb-e118-4c24-b257-79af1a95f629-kube-api-access-9mhnp\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826552    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-dshm\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826629 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826601    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-tmp-dir\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826629 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826609    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-home\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826731 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826629    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-model-cache\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.826879 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.826855    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-tmp-dir\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.828860 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.828837    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-dshm\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.829053 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.829036    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/165d32fb-e118-4c24-b257-79af1a95f629-tls-certs\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.833952 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.833930    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mhnp\" (UniqueName: \"kubernetes.io/projected/165d32fb-e118-4c24-b257-79af1a95f629-kube-api-access-9mhnp\") pod \"auth-test-model-3298cd-kserve-664cdbf675-9p9br\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") " pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:45.911904 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:45.911879    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:46.046858 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:46.046794    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"]
Jul 27 21:41:46.049046 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:41:46.049017    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod165d32fb_e118_4c24_b257_79af1a95f629.slice/crio-a2e2b3d95e59fd664673fc85b483ad9d113a8e64229876534f1341c03aebd242 WatchSource:0}: Error finding container a2e2b3d95e59fd664673fc85b483ad9d113a8e64229876534f1341c03aebd242: Status 404 returned error can't find the container with id a2e2b3d95e59fd664673fc85b483ad9d113a8e64229876534f1341c03aebd242
Jul 27 21:41:46.985451 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:46.985412    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br" event={"ID":"165d32fb-e118-4c24-b257-79af1a95f629","Type":"ContainerStarted","Data":"991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc"}
Jul 27 21:41:46.985825 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:46.985462    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br" event={"ID":"165d32fb-e118-4c24-b257-79af1a95f629","Type":"ContainerStarted","Data":"a2e2b3d95e59fd664673fc85b483ad9d113a8e64229876534f1341c03aebd242"}
Jul 27 21:41:47.002353 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:47.002301    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br" podStartSLOduration=2.002285605 podStartE2EDuration="2.002285605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:41:47.001133801 +0000 UTC m=+2650.304378565" watchObservedRunningTime="2026-07-27 21:41:47.002285605 +0000 UTC m=+2650.305530369"
Jul 27 21:41:55.912744 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:55.912691    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:55.913219 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:55.912761    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:55.924664 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:55.924638    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:41:56.038118 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:41:56.038090    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:42:15.001516 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.001480    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"]
Jul 27 21:42:15.004525 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.004507    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.007909 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.007888    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"openshift-service-ca.crt\""
Jul 27 21:42:15.008772 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.008753    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"auth-test-model-2cf6fc-kserve-self-signed-certs\""
Jul 27 21:42:15.008772 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.008756    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"kube-root-ca.crt\""
Jul 27 21:42:15.008947 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.008809    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"default-dockercfg-cpn4m\""
Jul 27 21:42:15.014957 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.014935    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"]
Jul 27 21:42:15.096186 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.096152    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz2b6\" (UniqueName: \"kubernetes.io/projected/2b6b1a39-6973-4b74-8fee-370f4f283cd8-kube-api-access-gz2b6\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.096362 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.096208    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-home\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.096362 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.096273    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-dshm\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.096362 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.096301    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tls-certs\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.096362 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.096351    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-model-cache\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.096513 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.096379    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tmp-dir\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.196869 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.196842    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-home\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.196869 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.196871    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-dshm\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197066 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.196887    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tls-certs\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197066 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.197015    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-model-cache\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197066 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.197060    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tmp-dir\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197200 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.197156    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz2b6\" (UniqueName: \"kubernetes.io/projected/2b6b1a39-6973-4b74-8fee-370f4f283cd8-kube-api-access-gz2b6\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197340 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.197320    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-home\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197457 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.197436    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-model-cache\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.197518 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.197467    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tmp-dir\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.199257 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.199214    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-dshm\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.199705 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.199689    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tls-certs\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.204668 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.204645    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz2b6\" (UniqueName: \"kubernetes.io/projected/2b6b1a39-6973-4b74-8fee-370f4f283cd8-kube-api-access-gz2b6\") pod \"auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") " pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.314197 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.314124    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:15.450427 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:15.450377    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"]
Jul 27 21:42:15.456018 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:42:15.455992    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b6b1a39_6973_4b74_8fee_370f4f283cd8.slice/crio-5a23ac56c765c475d60a49b18cdd3ff3db37ddc1ccf90f541930858379ff619a WatchSource:0}: Error finding container 5a23ac56c765c475d60a49b18cdd3ff3db37ddc1ccf90f541930858379ff619a: Status 404 returned error can't find the container with id 5a23ac56c765c475d60a49b18cdd3ff3db37ddc1ccf90f541930858379ff619a
Jul 27 21:42:16.113008 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:16.112975    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29" event={"ID":"2b6b1a39-6973-4b74-8fee-370f4f283cd8","Type":"ContainerStarted","Data":"8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e"}
Jul 27 21:42:16.113403 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:16.113015    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29" event={"ID":"2b6b1a39-6973-4b74-8fee-370f4f283cd8","Type":"ContainerStarted","Data":"5a23ac56c765c475d60a49b18cdd3ff3db37ddc1ccf90f541930858379ff619a"}
Jul 27 21:42:16.129927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:16.129877    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29" podStartSLOduration=2.1298618 podStartE2EDuration="2.1298618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:42:16.128746398 +0000 UTC m=+2679.431991164" watchObservedRunningTime="2026-07-27 21:42:16.1298618 +0000 UTC m=+2679.433106605"
Jul 27 21:42:25.314916 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:25.314885    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:25.315394 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:25.314929    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:25.327570 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:25.327551    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:42:26.166237 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:42:26.166194    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:46:36.010242 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:36.010200    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"]
Jul 27 21:46:36.844736 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:36.844705    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"]
Jul 27 21:46:37.346651 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:37.346618    2552 scope.go:117] "RemoveContainer" containerID="83efb27f78b21959d192d42b0f1836b80c21a0cfa83abeeba2fe28a2d3864022"
Jul 27 21:46:51.029265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.029197    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br" podUID="165d32fb-e118-4c24-b257-79af1a95f629" containerName="main" containerID="cri-o://991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc" gracePeriod=45
Jul 27 21:46:51.273175 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.273150    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:46:51.311972 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.311910    2552 generic.go:356] "Generic (PLEG): container finished" podID="165d32fb-e118-4c24-b257-79af1a95f629" containerID="991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc" exitCode=0
Jul 27 21:46:51.312104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.311967    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"
Jul 27 21:46:51.312104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.311981    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br" event={"ID":"165d32fb-e118-4c24-b257-79af1a95f629","Type":"ContainerDied","Data":"991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc"}
Jul 27 21:46:51.312104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.312013    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br" event={"ID":"165d32fb-e118-4c24-b257-79af1a95f629","Type":"ContainerDied","Data":"a2e2b3d95e59fd664673fc85b483ad9d113a8e64229876534f1341c03aebd242"}
Jul 27 21:46:51.312104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.312030    2552 scope.go:117] "RemoveContainer" containerID="991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc"
Jul 27 21:46:51.320878 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.320861    2552 scope.go:117] "RemoveContainer" containerID="991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc"
Jul 27 21:46:51.321149 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:46:51.321117    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc\": container with ID starting with 991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc not found: ID does not exist" containerID="991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc"
Jul 27 21:46:51.321265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.321158    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc"} err="failed to get container status \"991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc\": rpc error: code = NotFound desc = could not find container \"991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc\": container with ID starting with 991ccd5c7e425be7f2168d7caa73c3e465384f1a553f71fa8309f960f9baf6dc not found: ID does not exist"
Jul 27 21:46:51.360470 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360437    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-dshm\") pod \"165d32fb-e118-4c24-b257-79af1a95f629\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") "
Jul 27 21:46:51.360470 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360466    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-home\") pod \"165d32fb-e118-4c24-b257-79af1a95f629\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") "
Jul 27 21:46:51.360652 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360519    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-model-cache\") pod \"165d32fb-e118-4c24-b257-79af1a95f629\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") "
Jul 27 21:46:51.360652 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360541    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/165d32fb-e118-4c24-b257-79af1a95f629-tls-certs\") pod \"165d32fb-e118-4c24-b257-79af1a95f629\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") "
Jul 27 21:46:51.360652 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360588    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mhnp\" (UniqueName: \"kubernetes.io/projected/165d32fb-e118-4c24-b257-79af1a95f629-kube-api-access-9mhnp\") pod \"165d32fb-e118-4c24-b257-79af1a95f629\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") "
Jul 27 21:46:51.360652 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360615    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-tmp-dir\") pod \"165d32fb-e118-4c24-b257-79af1a95f629\" (UID: \"165d32fb-e118-4c24-b257-79af1a95f629\") "
Jul 27 21:46:51.360844 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360805    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-model-cache" (OuterVolumeSpecName: "model-cache") pod "165d32fb-e118-4c24-b257-79af1a95f629" (UID: "165d32fb-e118-4c24-b257-79af1a95f629"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:51.360844 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.360816    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-home" (OuterVolumeSpecName: "home") pod "165d32fb-e118-4c24-b257-79af1a95f629" (UID: "165d32fb-e118-4c24-b257-79af1a95f629"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:51.361022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.361000    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-home\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:51.361096 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.361017    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "165d32fb-e118-4c24-b257-79af1a95f629" (UID: "165d32fb-e118-4c24-b257-79af1a95f629"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:51.361096 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.361025    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-model-cache\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:51.362820 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.362796    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-dshm" (OuterVolumeSpecName: "dshm") pod "165d32fb-e118-4c24-b257-79af1a95f629" (UID: "165d32fb-e118-4c24-b257-79af1a95f629"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:51.362978 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.362952    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/165d32fb-e118-4c24-b257-79af1a95f629-kube-api-access-9mhnp" (OuterVolumeSpecName: "kube-api-access-9mhnp") pod "165d32fb-e118-4c24-b257-79af1a95f629" (UID: "165d32fb-e118-4c24-b257-79af1a95f629"). InnerVolumeSpecName "kube-api-access-9mhnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:46:51.363024 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.362972    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/165d32fb-e118-4c24-b257-79af1a95f629-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "165d32fb-e118-4c24-b257-79af1a95f629" (UID: "165d32fb-e118-4c24-b257-79af1a95f629"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:46:51.461590 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.461564    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-dshm\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:51.461590 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.461587    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/165d32fb-e118-4c24-b257-79af1a95f629-tls-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:51.461737 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.461598    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mhnp\" (UniqueName: \"kubernetes.io/projected/165d32fb-e118-4c24-b257-79af1a95f629-kube-api-access-9mhnp\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:51.461737 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.461606    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/165d32fb-e118-4c24-b257-79af1a95f629-tmp-dir\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:51.641575 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.641546    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"]
Jul 27 21:46:51.652050 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.652029    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/auth-test-model-3298cd-kserve-664cdbf675-9p9br"]
Jul 27 21:46:51.863798 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:51.863754    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29" podUID="2b6b1a39-6973-4b74-8fee-370f4f283cd8" containerName="main" containerID="cri-o://8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e" gracePeriod=45
Jul 27 21:46:52.105589 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.105565    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:46:52.269444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269415    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tls-certs\") pod \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") "
Jul 27 21:46:52.269586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269458    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-dshm\") pod \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") "
Jul 27 21:46:52.269586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269515    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-model-cache\") pod \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") "
Jul 27 21:46:52.269586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269540    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-home\") pod \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") "
Jul 27 21:46:52.269586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269560    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz2b6\" (UniqueName: \"kubernetes.io/projected/2b6b1a39-6973-4b74-8fee-370f4f283cd8-kube-api-access-gz2b6\") pod \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") "
Jul 27 21:46:52.269586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269575    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tmp-dir\") pod \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\" (UID: \"2b6b1a39-6973-4b74-8fee-370f4f283cd8\") "
Jul 27 21:46:52.269816 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269793    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-home" (OuterVolumeSpecName: "home") pod "2b6b1a39-6973-4b74-8fee-370f4f283cd8" (UID: "2b6b1a39-6973-4b74-8fee-370f4f283cd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:52.269874 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269805    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b6b1a39-6973-4b74-8fee-370f4f283cd8" (UID: "2b6b1a39-6973-4b74-8fee-370f4f283cd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:52.269928 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.269897    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b6b1a39-6973-4b74-8fee-370f4f283cd8" (UID: "2b6b1a39-6973-4b74-8fee-370f4f283cd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:52.271902 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.271879    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6b1a39-6973-4b74-8fee-370f4f283cd8-kube-api-access-gz2b6" (OuterVolumeSpecName: "kube-api-access-gz2b6") pod "2b6b1a39-6973-4b74-8fee-370f4f283cd8" (UID: "2b6b1a39-6973-4b74-8fee-370f4f283cd8"). InnerVolumeSpecName "kube-api-access-gz2b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:46:52.271902 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.271889    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b6b1a39-6973-4b74-8fee-370f4f283cd8" (UID: "2b6b1a39-6973-4b74-8fee-370f4f283cd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:46:52.272026 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.271933    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-dshm" (OuterVolumeSpecName: "dshm") pod "2b6b1a39-6973-4b74-8fee-370f4f283cd8" (UID: "2b6b1a39-6973-4b74-8fee-370f4f283cd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:46:52.317793 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.317768    2552 generic.go:356] "Generic (PLEG): container finished" podID="2b6b1a39-6973-4b74-8fee-370f4f283cd8" containerID="8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e" exitCode=0
Jul 27 21:46:52.317927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.317808    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"
Jul 27 21:46:52.317927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.317851    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29" event={"ID":"2b6b1a39-6973-4b74-8fee-370f4f283cd8","Type":"ContainerDied","Data":"8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e"}
Jul 27 21:46:52.317927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.317887    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29" event={"ID":"2b6b1a39-6973-4b74-8fee-370f4f283cd8","Type":"ContainerDied","Data":"5a23ac56c765c475d60a49b18cdd3ff3db37ddc1ccf90f541930858379ff619a"}
Jul 27 21:46:52.317927 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.317904    2552 scope.go:117] "RemoveContainer" containerID="8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e"
Jul 27 21:46:52.326304 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.326286    2552 scope.go:117] "RemoveContainer" containerID="8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e"
Jul 27 21:46:52.326538 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:46:52.326522    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e\": container with ID starting with 8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e not found: ID does not exist" containerID="8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e"
Jul 27 21:46:52.326588 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.326549    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e"} err="failed to get container status \"8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e\": rpc error: code = NotFound desc = could not find container \"8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e\": container with ID starting with 8998cf4cdd180b3bdfe53471b8455d22a768ab480810aa785c866fc51b05841e not found: ID does not exist"
Jul 27 21:46:52.348373 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.348347    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"]
Jul 27 21:46:52.359027 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.359008    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/auth-test-model-2cf6fc-kserve-77f6bb7c9f-hxt29"]
Jul 27 21:46:52.371120 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.371098    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tls-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:52.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.371122    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-dshm\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:52.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.371136    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-model-cache\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:52.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.371148    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-home\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:52.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.371161    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz2b6\" (UniqueName: \"kubernetes.io/projected/2b6b1a39-6973-4b74-8fee-370f4f283cd8-kube-api-access-gz2b6\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:52.371251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:52.371174    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b6b1a39-6973-4b74-8fee-370f4f283cd8-tmp-dir\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:46:53.198825 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:53.198789    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="165d32fb-e118-4c24-b257-79af1a95f629" path="/var/lib/kubelet/pods/165d32fb-e118-4c24-b257-79af1a95f629/volumes"
Jul 27 21:46:53.199191 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:46:53.199156    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b6b1a39-6973-4b74-8fee-370f4f283cd8" path="/var/lib/kubelet/pods/2b6b1a39-6973-4b74-8fee-370f4f283cd8/volumes"
Jul 27 21:47:27.424752 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.424719    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"]
Jul 27 21:47:27.425276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.425079    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b6b1a39-6973-4b74-8fee-370f4f283cd8" containerName="main"
Jul 27 21:47:27.425276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.425094    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b6b1a39-6973-4b74-8fee-370f4f283cd8" containerName="main"
Jul 27 21:47:27.425276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.425108    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="165d32fb-e118-4c24-b257-79af1a95f629" containerName="main"
Jul 27 21:47:27.425276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.425113    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="165d32fb-e118-4c24-b257-79af1a95f629" containerName="main"
Jul 27 21:47:27.425276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.425169    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b6b1a39-6973-4b74-8fee-370f4f283cd8" containerName="main"
Jul 27 21:47:27.425276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.425179    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="165d32fb-e118-4c24-b257-79af1a95f629" containerName="main"
Jul 27 21:47:27.427996 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.427981    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.432453 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.432432    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"openshift-service-ca.crt\""
Jul 27 21:47:27.432590 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.432570    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"sub-test-model-2cf6fc-kserve-self-signed-certs\""
Jul 27 21:47:27.432644 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.432620    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"kube-root-ca.crt\""
Jul 27 21:47:27.432691 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.432662    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-2cf6fc\"/\"default-dockercfg-cpn4m\""
Jul 27 21:47:27.444356 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.444330    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"]
Jul 27 21:47:27.457551 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.457531    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd5m6\" (UniqueName: \"kubernetes.io/projected/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-kube-api-access-qd5m6\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.457717 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.457574    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-home\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.457717 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.457596    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tmp-dir\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.457717 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.457630    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tls-certs\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.457717 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.457645    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-dshm\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.457717 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.457661    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-model-cache\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.558944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.558916    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd5m6\" (UniqueName: \"kubernetes.io/projected/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-kube-api-access-qd5m6\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.558962    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-home\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.558981    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tmp-dir\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.558995    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tls-certs\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.559014    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-dshm\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.559038    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-model-cache\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559442 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.559422    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-home\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559551 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.559503    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tmp-dir\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.559551 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.559537    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-model-cache\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.561498 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.561466    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-dshm\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.561727 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.561710    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tls-certs\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.573400 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.573377    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd5m6\" (UniqueName: \"kubernetes.io/projected/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-kube-api-access-qd5m6\") pod \"sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") " pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.737039 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.736970    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:27.879691 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:27.879662    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"]
Jul 27 21:47:27.887302 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:47:27.887276    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75090fcc_e7b9_4de1_bfc9_fb56bf81abd5.slice/crio-a06bf0126cca3ed8b57b3afcd9f3b1579f691949ce53d6b2de3ecb35b8b43e6f WatchSource:0}: Error finding container a06bf0126cca3ed8b57b3afcd9f3b1579f691949ce53d6b2de3ecb35b8b43e6f: Status 404 returned error can't find the container with id a06bf0126cca3ed8b57b3afcd9f3b1579f691949ce53d6b2de3ecb35b8b43e6f
Jul 27 21:47:28.480943 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:28.480910    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl" event={"ID":"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5","Type":"ContainerStarted","Data":"a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c"}
Jul 27 21:47:28.480943 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:28.480947    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl" event={"ID":"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5","Type":"ContainerStarted","Data":"a06bf0126cca3ed8b57b3afcd9f3b1579f691949ce53d6b2de3ecb35b8b43e6f"}
Jul 27 21:47:28.513626 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:28.513581    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl" podStartSLOduration=1.513567372 podStartE2EDuration="1.513567372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:47:28.510662246 +0000 UTC m=+2991.813907011" watchObservedRunningTime="2026-07-27 21:47:28.513567372 +0000 UTC m=+2991.816812137"
Jul 27 21:47:37.737739 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:37.737705    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:37.738110 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:37.737762    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:37.750524 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:37.750491    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:47:38.536427 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:47:38.536397    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:48:43.895862 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:43.895782    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"]
Jul 27 21:48:58.914705 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:58.914662    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl" podUID="75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" containerName="main" containerID="cri-o://a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c" gracePeriod=45
Jul 27 21:48:59.161537 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.161516    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:48:59.230473 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230407    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-model-cache\") pod \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") "
Jul 27 21:48:59.230473 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230460    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-home\") pod \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") "
Jul 27 21:48:59.230641 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230500    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd5m6\" (UniqueName: \"kubernetes.io/projected/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-kube-api-access-qd5m6\") pod \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") "
Jul 27 21:48:59.230641 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230518    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tls-certs\") pod \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") "
Jul 27 21:48:59.230641 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230545    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tmp-dir\") pod \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") "
Jul 27 21:48:59.230641 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230627    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-dshm\") pod \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\" (UID: \"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5\") "
Jul 27 21:48:59.230836 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230665    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-model-cache" (OuterVolumeSpecName: "model-cache") pod "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" (UID: "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:48:59.230836 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230779    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-home" (OuterVolumeSpecName: "home") pod "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" (UID: "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:48:59.230944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230885    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-model-cache\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:48:59.230944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230903    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-home\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:48:59.230944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.230891    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" (UID: "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:48:59.232736 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.232714    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-dshm" (OuterVolumeSpecName: "dshm") pod "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" (UID: "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:48:59.233200 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.233175    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" (UID: "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:48:59.233319 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.233203    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-kube-api-access-qd5m6" (OuterVolumeSpecName: "kube-api-access-qd5m6") pod "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" (UID: "75090fcc-e7b9-4de1-bfc9-fb56bf81abd5"). InnerVolumeSpecName "kube-api-access-qd5m6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:48:59.331446 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.331427    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-dshm\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:48:59.331446 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.331446    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd5m6\" (UniqueName: \"kubernetes.io/projected/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-kube-api-access-qd5m6\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:48:59.331586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.331457    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tls-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:48:59.331586 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.331467    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5-tmp-dir\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:48:59.872317 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.872278    2552 generic.go:356] "Generic (PLEG): container finished" podID="75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" containerID="a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c" exitCode=0
Jul 27 21:48:59.872491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.872327    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"
Jul 27 21:48:59.872491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.872363    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl" event={"ID":"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5","Type":"ContainerDied","Data":"a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c"}
Jul 27 21:48:59.872491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.872405    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl" event={"ID":"75090fcc-e7b9-4de1-bfc9-fb56bf81abd5","Type":"ContainerDied","Data":"a06bf0126cca3ed8b57b3afcd9f3b1579f691949ce53d6b2de3ecb35b8b43e6f"}
Jul 27 21:48:59.872491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.872423    2552 scope.go:117] "RemoveContainer" containerID="a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c"
Jul 27 21:48:59.881280 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.881263    2552 scope.go:117] "RemoveContainer" containerID="a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c"
Jul 27 21:48:59.881553 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:48:59.881535    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c\": container with ID starting with a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c not found: ID does not exist" containerID="a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c"
Jul 27 21:48:59.881590 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.881561    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c"} err="failed to get container status \"a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c\": rpc error: code = NotFound desc = could not find container \"a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c\": container with ID starting with a7b9b9ae6cf678619ac18ea75d3d41fc6ff935b2fc22e3c51180a05d1c9ac58c not found: ID does not exist"
Jul 27 21:48:59.893165 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.893145    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"]
Jul 27 21:48:59.896474 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:48:59.896454    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-2cf6fc/sub-test-model-2cf6fc-kserve-6b8f76ffdc-mp7bl"]
Jul 27 21:49:01.096335 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.096307    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"]
Jul 27 21:49:01.096693 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.096659    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" containerName="main"
Jul 27 21:49:01.096693 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.096670    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" containerName="main"
Jul 27 21:49:01.096759 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.096726    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" containerName="main"
Jul 27 21:49:01.100528 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.100510    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.103430 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.103406    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"kube-root-ca.crt\""
Jul 27 21:49:01.104499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.104481    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"rate-test-model-3298cd-kserve-self-signed-certs\""
Jul 27 21:49:01.104628 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.104487    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"openshift-service-ca.crt\""
Jul 27 21:49:01.104628 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.104593    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-3298cd\"/\"default-dockercfg-sfxzx\""
Jul 27 21:49:01.111898 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.111879    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"]
Jul 27 21:49:01.146665 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.146640    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-dshm\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.146770 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.146667    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-model-cache\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.146770 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.146710    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf72s\" (UniqueName: \"kubernetes.io/projected/a392ddf0-1792-4775-8c6d-f07c3739eb29-kube-api-access-wf72s\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.146848 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.146776    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-home\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.146848 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.146802    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a392ddf0-1792-4775-8c6d-f07c3739eb29-tls-certs\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.146914 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.146858    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-tmp-dir\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.197850 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.197821    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75090fcc-e7b9-4de1-bfc9-fb56bf81abd5" path="/var/lib/kubelet/pods/75090fcc-e7b9-4de1-bfc9-fb56bf81abd5/volumes"
Jul 27 21:49:01.247871 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.247843    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-tmp-dir\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248041 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248016    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-dshm\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248118 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248056    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-model-cache\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248175 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248131    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf72s\" (UniqueName: \"kubernetes.io/projected/a392ddf0-1792-4775-8c6d-f07c3739eb29-kube-api-access-wf72s\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248175 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248164    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-home\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248305 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248190    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a392ddf0-1792-4775-8c6d-f07c3739eb29-tls-certs\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248305 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248213    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-tmp-dir\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248423 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248406    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-model-cache\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.248554 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.248536    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-home\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.250379 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.250359    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-dshm\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.250653 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.250638    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a392ddf0-1792-4775-8c6d-f07c3739eb29-tls-certs\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.256272 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.256248    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf72s\" (UniqueName: \"kubernetes.io/projected/a392ddf0-1792-4775-8c6d-f07c3739eb29-kube-api-access-wf72s\") pod \"rate-test-model-3298cd-kserve-5cb9f57b79-scjww\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") " pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.410727 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.410704    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:01.545932 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.545877    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"]
Jul 27 21:49:01.548474 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:49:01.548449    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda392ddf0_1792_4775_8c6d_f07c3739eb29.slice/crio-33532a9e9682a0d734b74a96c2fbf4651d9847db7a805d63dd8101845ce18478 WatchSource:0}: Error finding container 33532a9e9682a0d734b74a96c2fbf4651d9847db7a805d63dd8101845ce18478: Status 404 returned error can't find the container with id 33532a9e9682a0d734b74a96c2fbf4651d9847db7a805d63dd8101845ce18478
Jul 27 21:49:01.883649 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.883567    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww" event={"ID":"a392ddf0-1792-4775-8c6d-f07c3739eb29","Type":"ContainerStarted","Data":"1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b"}
Jul 27 21:49:01.883649 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.883615    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww" event={"ID":"a392ddf0-1792-4775-8c6d-f07c3739eb29","Type":"ContainerStarted","Data":"33532a9e9682a0d734b74a96c2fbf4651d9847db7a805d63dd8101845ce18478"}
Jul 27 21:49:01.903637 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:01.903593    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww" podStartSLOduration=0.903576742 podStartE2EDuration="903.576742ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:49:01.902303447 +0000 UTC m=+3085.205548213" watchObservedRunningTime="2026-07-27 21:49:01.903576742 +0000 UTC m=+3085.206821538"
Jul 27 21:49:11.411119 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:11.411080    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:11.411633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:11.411134    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:11.423619 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:11.423596    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:49:11.940174 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:49:11.940149    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:50:29.867029 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:29.866996    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"]
Jul 27 21:50:34.596239 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.596197    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7480-140349426546992"]
Jul 27 21:50:34.600480 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.600457    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:34.602827 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.602804    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gh797\""
Jul 27 21:50:34.605766 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.605742    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7480-140349426546992"]
Jul 27 21:50:34.650770 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.650740    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mskj\" (UniqueName: \"kubernetes.io/projected/175f5199-7967-432c-8551-847d4d382d8c-kube-api-access-9mskj\") pod \"test-curl-7480-140349426546992\" (UID: \"175f5199-7967-432c-8551-847d4d382d8c\") " pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:34.751519 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.751496    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mskj\" (UniqueName: \"kubernetes.io/projected/175f5199-7967-432c-8551-847d4d382d8c-kube-api-access-9mskj\") pod \"test-curl-7480-140349426546992\" (UID: \"175f5199-7967-432c-8551-847d4d382d8c\") " pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:34.759684 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.759661    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mskj\" (UniqueName: \"kubernetes.io/projected/175f5199-7967-432c-8551-847d4d382d8c-kube-api-access-9mskj\") pod \"test-curl-7480-140349426546992\" (UID: \"175f5199-7967-432c-8551-847d4d382d8c\") " pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:34.911498 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:34.911464    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:35.049810 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:35.049777    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7480-140349426546992"]
Jul 27 21:50:35.052413 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:50:35.052388    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod175f5199_7967_432c_8551_847d4d382d8c.slice/crio-1e8aa66fe1ad81103bd1b6731330bfc68e91ac568e46c351c3067ce2166fa231 WatchSource:0}: Error finding container 1e8aa66fe1ad81103bd1b6731330bfc68e91ac568e46c351c3067ce2166fa231: Status 404 returned error can't find the container with id 1e8aa66fe1ad81103bd1b6731330bfc68e91ac568e46c351c3067ce2166fa231
Jul 27 21:50:35.054290 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:35.054269    2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 27 21:50:35.296016 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:35.295932    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7480-140349426546992" event={"ID":"175f5199-7967-432c-8551-847d4d382d8c","Type":"ContainerStarted","Data":"1e8aa66fe1ad81103bd1b6731330bfc68e91ac568e46c351c3067ce2166fa231"}
Jul 27 21:50:36.302009 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:36.301976    2552 generic.go:356] "Generic (PLEG): container finished" podID="175f5199-7967-432c-8551-847d4d382d8c" containerID="edbd0f0d00aec3a3b05c1102050996016be6b4538bbc07633fff795b0702ec4a" exitCode=0
Jul 27 21:50:36.302465 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:36.302052    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7480-140349426546992" event={"ID":"175f5199-7967-432c-8551-847d4d382d8c","Type":"ContainerDied","Data":"edbd0f0d00aec3a3b05c1102050996016be6b4538bbc07633fff795b0702ec4a"}
Jul 27 21:50:37.436124 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.436096    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:37.474353 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.474296    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7480-140349426546992_175f5199-7967-432c-8551-847d4d382d8c/test-curl-7480-140349426546992/0.log"
Jul 27 21:50:37.475385 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.475367    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mskj\" (UniqueName: \"kubernetes.io/projected/175f5199-7967-432c-8551-847d4d382d8c-kube-api-access-9mskj\") pod \"175f5199-7967-432c-8551-847d4d382d8c\" (UID: \"175f5199-7967-432c-8551-847d4d382d8c\") "
Jul 27 21:50:37.477864 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.477846    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175f5199-7967-432c-8551-847d4d382d8c-kube-api-access-9mskj" (OuterVolumeSpecName: "kube-api-access-9mskj") pod "175f5199-7967-432c-8551-847d4d382d8c" (UID: "175f5199-7967-432c-8551-847d4d382d8c"). InnerVolumeSpecName "kube-api-access-9mskj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:50:37.499135 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.499113    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7480-140349426546992"]
Jul 27 21:50:37.503140 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.503120    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7480-140349426546992"]
Jul 27 21:50:37.576080 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.576029    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mskj\" (UniqueName: \"kubernetes.io/projected/175f5199-7967-432c-8551-847d4d382d8c-kube-api-access-9mskj\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:37.665863 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.665839    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7480-140349431523200"]
Jul 27 21:50:37.666192 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.666179    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="175f5199-7967-432c-8551-847d4d382d8c" containerName="test-curl-7480-140349426546992"
Jul 27 21:50:37.666260 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.666193    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="175f5199-7967-432c-8551-847d4d382d8c" containerName="test-curl-7480-140349426546992"
Jul 27 21:50:37.666293 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.666281    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="175f5199-7967-432c-8551-847d4d382d8c" containerName="test-curl-7480-140349426546992"
Jul 27 21:50:37.669201 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.669188    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:37.676410 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.676382    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7480-140349431523200"]
Jul 27 21:50:37.777118 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.777087    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l75q6\" (UniqueName: \"kubernetes.io/projected/f9148e48-2167-478f-8054-f1afe36750b2-kube-api-access-l75q6\") pod \"test-curl-7480-140349431523200\" (UID: \"f9148e48-2167-478f-8054-f1afe36750b2\") " pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:37.878050 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.877988    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75q6\" (UniqueName: \"kubernetes.io/projected/f9148e48-2167-478f-8054-f1afe36750b2-kube-api-access-l75q6\") pod \"test-curl-7480-140349431523200\" (UID: \"f9148e48-2167-478f-8054-f1afe36750b2\") " pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:37.886802 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.886774    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l75q6\" (UniqueName: \"kubernetes.io/projected/f9148e48-2167-478f-8054-f1afe36750b2-kube-api-access-l75q6\") pod \"test-curl-7480-140349431523200\" (UID: \"f9148e48-2167-478f-8054-f1afe36750b2\") " pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:37.978576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:37.978552    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:38.100890 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:38.100863    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7480-140349431523200"]
Jul 27 21:50:38.102468 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:50:38.102428    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9148e48_2167_478f_8054_f1afe36750b2.slice/crio-eef56ef04cfeabc15c3e7774312de46706c7fc10afa086c30dec6dd0e70bd8ec WatchSource:0}: Error finding container eef56ef04cfeabc15c3e7774312de46706c7fc10afa086c30dec6dd0e70bd8ec: Status 404 returned error can't find the container with id eef56ef04cfeabc15c3e7774312de46706c7fc10afa086c30dec6dd0e70bd8ec
Jul 27 21:50:38.311993 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:38.311965    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:38.311993 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:38.311981    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e8aa66fe1ad81103bd1b6731330bfc68e91ac568e46c351c3067ce2166fa231"
Jul 27 21:50:38.313057 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:38.313035    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7480-140349431523200" event={"ID":"f9148e48-2167-478f-8054-f1afe36750b2","Type":"ContainerStarted","Data":"eef56ef04cfeabc15c3e7774312de46706c7fc10afa086c30dec6dd0e70bd8ec"}
Jul 27 21:50:38.322397 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:50:38.322372    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7480-140349426546992\" is forbidden: User \"system:node:ip-10-0-140-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-33.ec2.internal' and this object" podUID="175f5199-7967-432c-8551-847d4d382d8c" pod="opendatahub/test-curl-7480-140349426546992"
Jul 27 21:50:39.198039 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:39.198005    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="175f5199-7967-432c-8551-847d4d382d8c" path="/var/lib/kubelet/pods/175f5199-7967-432c-8551-847d4d382d8c/volumes"
Jul 27 21:50:39.318741 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:39.318714    2552 generic.go:356] "Generic (PLEG): container finished" podID="f9148e48-2167-478f-8054-f1afe36750b2" containerID="48ffb35dc918ba0de3e98d0f66bf93fbf4650f11f5746f75636552856c62106c" exitCode=0
Jul 27 21:50:39.318904 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:39.318804    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7480-140349431523200" event={"ID":"f9148e48-2167-478f-8054-f1afe36750b2","Type":"ContainerDied","Data":"48ffb35dc918ba0de3e98d0f66bf93fbf4650f11f5746f75636552856c62106c"}
Jul 27 21:50:40.456501 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.456478    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:40.479216 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.479187    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7480-140349431523200_f9148e48-2167-478f-8054-f1afe36750b2/test-curl-7480-140349431523200/0.log"
Jul 27 21:50:40.500571 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.500548    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l75q6\" (UniqueName: \"kubernetes.io/projected/f9148e48-2167-478f-8054-f1afe36750b2-kube-api-access-l75q6\") pod \"f9148e48-2167-478f-8054-f1afe36750b2\" (UID: \"f9148e48-2167-478f-8054-f1afe36750b2\") "
Jul 27 21:50:40.502658 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.502627    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9148e48-2167-478f-8054-f1afe36750b2-kube-api-access-l75q6" (OuterVolumeSpecName: "kube-api-access-l75q6") pod "f9148e48-2167-478f-8054-f1afe36750b2" (UID: "f9148e48-2167-478f-8054-f1afe36750b2"). InnerVolumeSpecName "kube-api-access-l75q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:50:40.506387 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.506365    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7480-140349431523200"]
Jul 27 21:50:40.511015 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.510995    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7480-140349431523200"]
Jul 27 21:50:40.601263 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:40.601197    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l75q6\" (UniqueName: \"kubernetes.io/projected/f9148e48-2167-478f-8054-f1afe36750b2-kube-api-access-l75q6\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:41.197892 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:41.197857    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9148e48-2167-478f-8054-f1afe36750b2" path="/var/lib/kubelet/pods/f9148e48-2167-478f-8054-f1afe36750b2/volumes"
Jul 27 21:50:41.334058 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:41.333996    2552 scope.go:117] "RemoveContainer" containerID="48ffb35dc918ba0de3e98d0f66bf93fbf4650f11f5746f75636552856c62106c"
Jul 27 21:50:41.334248 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:41.334074    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349431523200"
Jul 27 21:50:44.886529 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:44.886488    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww" podUID="a392ddf0-1792-4775-8c6d-f07c3739eb29" containerName="main" containerID="cri-o://1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b" gracePeriod=45
Jul 27 21:50:45.135381 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.135354    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:50:45.237827 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.237797    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf72s\" (UniqueName: \"kubernetes.io/projected/a392ddf0-1792-4775-8c6d-f07c3739eb29-kube-api-access-wf72s\") pod \"a392ddf0-1792-4775-8c6d-f07c3739eb29\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") "
Jul 27 21:50:45.237997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.237850    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-model-cache\") pod \"a392ddf0-1792-4775-8c6d-f07c3739eb29\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") "
Jul 27 21:50:45.237997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.237871    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-home\") pod \"a392ddf0-1792-4775-8c6d-f07c3739eb29\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") "
Jul 27 21:50:45.237997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.237961    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-tmp-dir\") pod \"a392ddf0-1792-4775-8c6d-f07c3739eb29\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") "
Jul 27 21:50:45.237997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.237985    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-dshm\") pod \"a392ddf0-1792-4775-8c6d-f07c3739eb29\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") "
Jul 27 21:50:45.238191 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238021    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a392ddf0-1792-4775-8c6d-f07c3739eb29-tls-certs\") pod \"a392ddf0-1792-4775-8c6d-f07c3739eb29\" (UID: \"a392ddf0-1792-4775-8c6d-f07c3739eb29\") "
Jul 27 21:50:45.238191 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238144    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-model-cache" (OuterVolumeSpecName: "model-cache") pod "a392ddf0-1792-4775-8c6d-f07c3739eb29" (UID: "a392ddf0-1792-4775-8c6d-f07c3739eb29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:50:45.238323 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238217    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-home" (OuterVolumeSpecName: "home") pod "a392ddf0-1792-4775-8c6d-f07c3739eb29" (UID: "a392ddf0-1792-4775-8c6d-f07c3739eb29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:50:45.238323 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238255    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a392ddf0-1792-4775-8c6d-f07c3739eb29" (UID: "a392ddf0-1792-4775-8c6d-f07c3739eb29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:50:45.238323 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238317    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-tmp-dir\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:45.238444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238329    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-model-cache\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:45.238444 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.238339    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-home\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:45.240064 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.240040    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a392ddf0-1792-4775-8c6d-f07c3739eb29-kube-api-access-wf72s" (OuterVolumeSpecName: "kube-api-access-wf72s") pod "a392ddf0-1792-4775-8c6d-f07c3739eb29" (UID: "a392ddf0-1792-4775-8c6d-f07c3739eb29"). InnerVolumeSpecName "kube-api-access-wf72s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:50:45.240161 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.240143    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-dshm" (OuterVolumeSpecName: "dshm") pod "a392ddf0-1792-4775-8c6d-f07c3739eb29" (UID: "a392ddf0-1792-4775-8c6d-f07c3739eb29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:50:45.240262 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.240214    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a392ddf0-1792-4775-8c6d-f07c3739eb29-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a392ddf0-1792-4775-8c6d-f07c3739eb29" (UID: "a392ddf0-1792-4775-8c6d-f07c3739eb29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:50:45.339784 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.339742    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a392ddf0-1792-4775-8c6d-f07c3739eb29-dshm\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:45.339784 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.339773    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a392ddf0-1792-4775-8c6d-f07c3739eb29-tls-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:45.339953 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.339791    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf72s\" (UniqueName: \"kubernetes.io/projected/a392ddf0-1792-4775-8c6d-f07c3739eb29-kube-api-access-wf72s\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:45.355997 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.355969    2552 generic.go:356] "Generic (PLEG): container finished" podID="a392ddf0-1792-4775-8c6d-f07c3739eb29" containerID="1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b" exitCode=0
Jul 27 21:50:45.356089 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.356014    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"
Jul 27 21:50:45.356089 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.356016    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww" event={"ID":"a392ddf0-1792-4775-8c6d-f07c3739eb29","Type":"ContainerDied","Data":"1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b"}
Jul 27 21:50:45.356089 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.356060    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww" event={"ID":"a392ddf0-1792-4775-8c6d-f07c3739eb29","Type":"ContainerDied","Data":"33532a9e9682a0d734b74a96c2fbf4651d9847db7a805d63dd8101845ce18478"}
Jul 27 21:50:45.356089 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.356078    2552 scope.go:117] "RemoveContainer" containerID="1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b"
Jul 27 21:50:45.366638 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.366612    2552 scope.go:117] "RemoveContainer" containerID="1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b"
Jul 27 21:50:45.367101 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:50:45.367083    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b\": container with ID starting with 1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b not found: ID does not exist" containerID="1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b"
Jul 27 21:50:45.367168 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.367109    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b"} err="failed to get container status \"1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b\": rpc error: code = NotFound desc = could not find container \"1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b\": container with ID starting with 1a2b5212430e58370788c72ec108fa4a44fa560bf6788951013ca2d589ff2d9b not found: ID does not exist"
Jul 27 21:50:45.376580 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.376561    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"]
Jul 27 21:50:45.382099 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:45.382077    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-3298cd/rate-test-model-3298cd-kserve-5cb9f57b79-scjww"]
Jul 27 21:50:46.698516 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698479    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7480-140349426513504"]
Jul 27 21:50:46.698882 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698841    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a392ddf0-1792-4775-8c6d-f07c3739eb29" containerName="main"
Jul 27 21:50:46.698882 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698855    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="a392ddf0-1792-4775-8c6d-f07c3739eb29" containerName="main"
Jul 27 21:50:46.698882 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698874    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9148e48-2167-478f-8054-f1afe36750b2" containerName="test-curl-7480-140349431523200"
Jul 27 21:50:46.698882 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698880    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9148e48-2167-478f-8054-f1afe36750b2" containerName="test-curl-7480-140349431523200"
Jul 27 21:50:46.699020 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698941    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="a392ddf0-1792-4775-8c6d-f07c3739eb29" containerName="main"
Jul 27 21:50:46.699020 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.698951    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9148e48-2167-478f-8054-f1afe36750b2" containerName="test-curl-7480-140349431523200"
Jul 27 21:50:46.701266 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.701247    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:46.703902 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.703880    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gh797\""
Jul 27 21:50:46.705514 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.705491    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7480-140349426513504"]
Jul 27 21:50:46.751701 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.751674    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d45mp\" (UniqueName: \"kubernetes.io/projected/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac-kube-api-access-d45mp\") pod \"test-curl-7480-140349426513504\" (UID: \"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac\") " pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:46.852333 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.852306    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d45mp\" (UniqueName: \"kubernetes.io/projected/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac-kube-api-access-d45mp\") pod \"test-curl-7480-140349426513504\" (UID: \"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac\") " pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:46.861193 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:46.861172    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d45mp\" (UniqueName: \"kubernetes.io/projected/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac-kube-api-access-d45mp\") pod \"test-curl-7480-140349426513504\" (UID: \"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac\") " pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:47.011437 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:47.011367    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:47.142140 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:47.142104    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7480-140349426513504"]
Jul 27 21:50:47.144993 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:50:47.144963    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cef1bb1_f80d_4b13_9c73_9cbae0be6dac.slice/crio-2470c05a2f8483bcfbd76e66f32e77b0cae4f3061222ff5d1349687d459fb82b WatchSource:0}: Error finding container 2470c05a2f8483bcfbd76e66f32e77b0cae4f3061222ff5d1349687d459fb82b: Status 404 returned error can't find the container with id 2470c05a2f8483bcfbd76e66f32e77b0cae4f3061222ff5d1349687d459fb82b
Jul 27 21:50:47.197517 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:47.197491    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a392ddf0-1792-4775-8c6d-f07c3739eb29" path="/var/lib/kubelet/pods/a392ddf0-1792-4775-8c6d-f07c3739eb29/volumes"
Jul 27 21:50:47.368413 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:47.368326    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7480-140349426513504" event={"ID":"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac","Type":"ContainerStarted","Data":"2470c05a2f8483bcfbd76e66f32e77b0cae4f3061222ff5d1349687d459fb82b"}
Jul 27 21:50:48.373345 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:48.373313    2552 generic.go:356] "Generic (PLEG): container finished" podID="3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" containerID="ee0de095d0013ad20acfa0b3eaecf4208a938cff3bd6636f567e289b98c77149" exitCode=0
Jul 27 21:50:48.373750 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:48.373396    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7480-140349426513504" event={"ID":"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac","Type":"ContainerDied","Data":"ee0de095d0013ad20acfa0b3eaecf4208a938cff3bd6636f567e289b98c77149"}
Jul 27 21:50:49.504611 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.504587    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:49.526977 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.526950    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7480-140349426513504_3cef1bb1-f80d-4b13-9c73-9cbae0be6dac/test-curl-7480-140349426513504/0.log"
Jul 27 21:50:49.549666 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.549638    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7480-140349426513504"]
Jul 27 21:50:49.551875 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.551853    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7480-140349426513504"]
Jul 27 21:50:49.575753 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.575732    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d45mp\" (UniqueName: \"kubernetes.io/projected/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac-kube-api-access-d45mp\") pod \"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac\" (UID: \"3cef1bb1-f80d-4b13-9c73-9cbae0be6dac\") "
Jul 27 21:50:49.577863 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.577840    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac-kube-api-access-d45mp" (OuterVolumeSpecName: "kube-api-access-d45mp") pod "3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" (UID: "3cef1bb1-f80d-4b13-9c73-9cbae0be6dac"). InnerVolumeSpecName "kube-api-access-d45mp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:50:49.676251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:49.676213    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d45mp\" (UniqueName: \"kubernetes.io/projected/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac-kube-api-access-d45mp\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:50:50.382964 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:50.382933    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:50.383151 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:50.382937    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2470c05a2f8483bcfbd76e66f32e77b0cae4f3061222ff5d1349687d459fb82b"
Jul 27 21:50:50.392367 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:50:50.392341    2552 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7480-140349426513504\" is forbidden: User \"system:node:ip-10-0-140-33.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-33.ec2.internal' and this object" podUID="3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" pod="opendatahub/test-curl-7480-140349426513504"
Jul 27 21:50:51.198184 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:50:51.198147    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" path="/var/lib/kubelet/pods/3cef1bb1-f80d-4b13-9c73-9cbae0be6dac/volumes"
Jul 27 21:51:12.645370 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.645336    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"]
Jul 27 21:51:12.645954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.645794    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" containerName="test-curl-7480-140349426513504"
Jul 27 21:51:12.645954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.645812    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" containerName="test-curl-7480-140349426513504"
Jul 27 21:51:12.645954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.645905    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cef1bb1-f80d-4b13-9c73-9cbae0be6dac" containerName="test-curl-7480-140349426513504"
Jul 27 21:51:12.649656 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.649634    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.652325 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.652303    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c3af52-serving-cert\""
Jul 27 21:51:12.652431 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.652379    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 21:51:12.652431 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.652398    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9gdtk\""
Jul 27 21:51:12.653365 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.653348    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 27 21:51:12.663690 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.663669    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"]
Jul 27 21:51:12.671896 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.671875    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.671999 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.671921    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.672067 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.672023    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvn7l\" (UniqueName: \"kubernetes.io/projected/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-kube-api-access-xvn7l\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.772611 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.772584    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvn7l\" (UniqueName: \"kubernetes.io/projected/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-kube-api-access-xvn7l\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.772745 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.772632    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.772745 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.772689    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.773292 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.773275    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.775152 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.775131    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.782144 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.782120    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvn7l\" (UniqueName: \"kubernetes.io/projected/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-kube-api-access-xvn7l\") pod \"maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:12.960163 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:12.960140    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:13.122085 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.122053    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"]
Jul 27 21:51:13.125097 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:51:13.125068    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d94ec0_26a7_43e1_8cb2_3774a67bbe7d.slice/crio-961bdccaaa002e59c4cb42df745840fbe62ecb466f3d31cc3cf71d0431005d2d WatchSource:0}: Error finding container 961bdccaaa002e59c4cb42df745840fbe62ecb466f3d31cc3cf71d0431005d2d: Status 404 returned error can't find the container with id 961bdccaaa002e59c4cb42df745840fbe62ecb466f3d31cc3cf71d0431005d2d
Jul 27 21:51:13.483033 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.482997    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds" event={"ID":"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d","Type":"ContainerStarted","Data":"961bdccaaa002e59c4cb42df745840fbe62ecb466f3d31cc3cf71d0431005d2d"}
Jul 27 21:51:13.542405 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.542372    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"]
Jul 27 21:51:13.545211 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.545195    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.547671 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.547645    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c3af52\""
Jul 27 21:51:13.547874 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.547861    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c3af52-dockercfg-dnkd2\""
Jul 27 21:51:13.557694 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.557675    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"]
Jul 27 21:51:13.579208 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.579186    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdrd7\" (UniqueName: \"kubernetes.io/projected/ddc425a3-6406-4ef4-929b-da752821b461-kube-api-access-gdrd7\") pod \"payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.579326 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.579250    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddc425a3-6406-4ef4-929b-da752821b461-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.680680 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.680622    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdrd7\" (UniqueName: \"kubernetes.io/projected/ddc425a3-6406-4ef4-929b-da752821b461-kube-api-access-gdrd7\") pod \"payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.681123 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.680736    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddc425a3-6406-4ef4-929b-da752821b461-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.681609 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.681585    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddc425a3-6406-4ef4-929b-da752821b461-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.691348 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.691328    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdrd7\" (UniqueName: \"kubernetes.io/projected/ddc425a3-6406-4ef4-929b-da752821b461-kube-api-access-gdrd7\") pod \"payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:13.856819 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:13.856537    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:14.023824 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:14.023795    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"]
Jul 27 21:51:14.026357 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:51:14.026282    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddc425a3_6406_4ef4_929b_da752821b461.slice/crio-e9f9f31147ac6900105db1dc45ecb53756ad24a3714e9013519fde0247acf9fb WatchSource:0}: Error finding container e9f9f31147ac6900105db1dc45ecb53756ad24a3714e9013519fde0247acf9fb: Status 404 returned error can't find the container with id e9f9f31147ac6900105db1dc45ecb53756ad24a3714e9013519fde0247acf9fb
Jul 27 21:51:14.491329 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:14.491287    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6" event={"ID":"ddc425a3-6406-4ef4-929b-da752821b461","Type":"ContainerStarted","Data":"e9f9f31147ac6900105db1dc45ecb53756ad24a3714e9013519fde0247acf9fb"}
Jul 27 21:51:15.498868 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:15.498834    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds" event={"ID":"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d","Type":"ContainerStarted","Data":"acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb"}
Jul 27 21:51:15.499210 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:15.498975    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:15.515915 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:15.515866    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds" podStartSLOduration=1.235479532 podStartE2EDuration="3.515845534s" totalImagesPullingTime="2.280366002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:51:15.514196041 +0000 UTC m=+3218.817440811" watchObservedRunningTime="2026-07-27 21:51:15.515845534 +0000 UTC m=+3218.819090301"
Jul 27 21:51:17.510786 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:17.510692    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6" event={"ID":"ddc425a3-6406-4ef4-929b-da752821b461","Type":"ContainerStarted","Data":"7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929"}
Jul 27 21:51:17.511249 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:17.510835    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:17.528356 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:17.528300    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6" podStartSLOduration=1.383168695 podStartE2EDuration="4.528284588s" totalImagesPullingTime="3.145115893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:51:17.527425592 +0000 UTC m=+3220.830670361" watchObservedRunningTime="2026-07-27 21:51:17.528284588 +0000 UTC m=+3220.831529385"
Jul 27 21:51:21.509866 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:21.509840    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:51:28.518200 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:28.518126    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:51:53.392518 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.392442    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"]
Jul 27 21:51:53.395796 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.395780    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.398220 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.398198    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-262475\"/\"kube-root-ca.crt\""
Jul 27 21:51:53.399219 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.399196    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-262475\"/\"default-dockercfg-w5nsg\""
Jul 27 21:51:53.399352 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.399324    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-262475\"/\"openshift-service-ca.crt\""
Jul 27 21:51:53.399461 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.399440    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-262475\"/\"ipp-route-262475-kserve-self-signed-certs\""
Jul 27 21:51:53.404672 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.404652    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"]
Jul 27 21:51:53.515440 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.515409    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-model-cache\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.515628 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.515469    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q65s\" (UniqueName: \"kubernetes.io/projected/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-kube-api-access-5q65s\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.515628 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.515549    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tls-certs\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.515628 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.515586    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-dshm\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.515628 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.515608    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tmp-dir\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.515794 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.515643    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-home\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616495 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616465    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-home\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616526    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-model-cache\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616561    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q65s\" (UniqueName: \"kubernetes.io/projected/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-kube-api-access-5q65s\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616591    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tls-certs\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616608    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-dshm\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616623    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tmp-dir\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.616941 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616921    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-home\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.617009 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616961    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tmp-dir\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.617009 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.616999    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-model-cache\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.618993 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.618966    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-dshm\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.619491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.619463    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tls-certs\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.625362 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.625339    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q65s\" (UniqueName: \"kubernetes.io/projected/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-kube-api-access-5q65s\") pod \"ipp-route-262475-kserve-6d67f7bd67-trqm6\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") " pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.705286 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.705259    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:51:53.844522 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:53.844491    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"]
Jul 27 21:51:53.847385 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:51:53.847359    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd551e53_38cf_4ee3_a99a_25a91bfe6c8e.slice/crio-e776017f40e5aeafb01aa89cd7476ecc90934702da3e9b4a5df6535f1e64a8a9 WatchSource:0}: Error finding container e776017f40e5aeafb01aa89cd7476ecc90934702da3e9b4a5df6535f1e64a8a9: Status 404 returned error can't find the container with id e776017f40e5aeafb01aa89cd7476ecc90934702da3e9b4a5df6535f1e64a8a9
Jul 27 21:51:54.677939 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:54.677897    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6" event={"ID":"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e","Type":"ContainerStarted","Data":"10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8"}
Jul 27 21:51:54.678329 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:54.677949    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6" event={"ID":"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e","Type":"ContainerStarted","Data":"e776017f40e5aeafb01aa89cd7476ecc90934702da3e9b4a5df6535f1e64a8a9"}
Jul 27 21:51:54.695301 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:51:54.695245    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6" podStartSLOduration=1.6952058110000001 podStartE2EDuration="1.695205811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:51:54.694185014 +0000 UTC m=+3257.997429791" watchObservedRunningTime="2026-07-27 21:51:54.695205811 +0000 UTC m=+3257.998450575"
Jul 27 21:52:03.706354 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:03.706314    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:52:03.706760 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:03.706372    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:52:03.719519 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:03.719489    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:52:03.730800 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:03.730782    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:52:41.708557 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.708518    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"]
Jul 27 21:52:41.711378 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.711355    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.713737 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.713717    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-ad54ee-serving-cert\""
Jul 27 21:52:41.724266 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.724215    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"]
Jul 27 21:52:41.830579 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.830547    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6727b17-30a0-4170-b4e8-e04928ef327e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.830720 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.830607    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm74f\" (UniqueName: \"kubernetes.io/projected/c6727b17-30a0-4170-b4e8-e04928ef327e-kube-api-access-wm74f\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.830720 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.830646    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6727b17-30a0-4170-b4e8-e04928ef327e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.931438 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.931410    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm74f\" (UniqueName: \"kubernetes.io/projected/c6727b17-30a0-4170-b4e8-e04928ef327e-kube-api-access-wm74f\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.931576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.931447    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6727b17-30a0-4170-b4e8-e04928ef327e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.931576 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.931510    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6727b17-30a0-4170-b4e8-e04928ef327e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.932114 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.932090    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6727b17-30a0-4170-b4e8-e04928ef327e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.933864 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.933839    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6727b17-30a0-4170-b4e8-e04928ef327e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:41.940749 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:41.940726    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm74f\" (UniqueName: \"kubernetes.io/projected/c6727b17-30a0-4170-b4e8-e04928ef327e-kube-api-access-wm74f\") pod \"maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:42.022295 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.022214    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:42.160033 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.160009    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"]
Jul 27 21:52:42.162243 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:52:42.162205    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6727b17_30a0_4170_b4e8_e04928ef327e.slice/crio-ca13ecbb1a72f97d1ee598f7df17c4fd8296968439b128d7657b082fb7bf1f37 WatchSource:0}: Error finding container ca13ecbb1a72f97d1ee598f7df17c4fd8296968439b128d7657b082fb7bf1f37: Status 404 returned error can't find the container with id ca13ecbb1a72f97d1ee598f7df17c4fd8296968439b128d7657b082fb7bf1f37
Jul 27 21:52:42.332503 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.332427    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"]
Jul 27 21:52:42.335443 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.335424    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.338115 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.338092    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-ad54ee-dockercfg-zklqk\""
Jul 27 21:52:42.338205 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.338139    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-ad54ee\""
Jul 27 21:52:42.349697 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.349678    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"]
Jul 27 21:52:42.351881 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.351860    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"]
Jul 27 21:52:42.351964 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.351948    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.365895 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.365871    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"]
Jul 27 21:52:42.435738 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.435708    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e582ba3-399e-476f-9483-b5df79cc2ec2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.435885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.435750    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsxlj\" (UniqueName: \"kubernetes.io/projected/dc7cf966-3fa4-4019-a423-66ebf89bcff6-kube-api-access-wsxlj\") pod \"payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.435885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.435775    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpnm4\" (UniqueName: \"kubernetes.io/projected/9e582ba3-399e-476f-9483-b5df79cc2ec2-kube-api-access-rpnm4\") pod \"payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.435992 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.435959    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc7cf966-3fa4-4019-a423-66ebf89bcff6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.536731 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.536695    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpnm4\" (UniqueName: \"kubernetes.io/projected/9e582ba3-399e-476f-9483-b5df79cc2ec2-kube-api-access-rpnm4\") pod \"payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.536894 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.536794    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc7cf966-3fa4-4019-a423-66ebf89bcff6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.536894 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.536862    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e582ba3-399e-476f-9483-b5df79cc2ec2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.537011 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.536899    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsxlj\" (UniqueName: \"kubernetes.io/projected/dc7cf966-3fa4-4019-a423-66ebf89bcff6-kube-api-access-wsxlj\") pod \"payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.537609 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.537587    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e582ba3-399e-476f-9483-b5df79cc2ec2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.537690 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.537628    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc7cf966-3fa4-4019-a423-66ebf89bcff6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.544706 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.544680    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsxlj\" (UniqueName: \"kubernetes.io/projected/dc7cf966-3fa4-4019-a423-66ebf89bcff6-kube-api-access-wsxlj\") pod \"payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.544795 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.544712    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpnm4\" (UniqueName: \"kubernetes.io/projected/9e582ba3-399e-476f-9483-b5df79cc2ec2-kube-api-access-rpnm4\") pod \"payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.647507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.647480    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.661596 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.661573    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.793371 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.792553    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"]
Jul 27 21:52:42.798171 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:52:42.798136    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e582ba3_399e_476f_9483_b5df79cc2ec2.slice/crio-27eccb3a5baa1d3d5e0cb3fd152163bd4db5ea268ef910e48e2519b7da749079 WatchSource:0}: Error finding container 27eccb3a5baa1d3d5e0cb3fd152163bd4db5ea268ef910e48e2519b7da749079: Status 404 returned error can't find the container with id 27eccb3a5baa1d3d5e0cb3fd152163bd4db5ea268ef910e48e2519b7da749079
Jul 27 21:52:42.813671 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.813639    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"]
Jul 27 21:52:42.817431 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:52:42.817409    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7cf966_3fa4_4019_a423_66ebf89bcff6.slice/crio-45065e98edf0b5d7e6425deda71445fcfe00613bd6726ff85229c0ec43448805 WatchSource:0}: Error finding container 45065e98edf0b5d7e6425deda71445fcfe00613bd6726ff85229c0ec43448805: Status 404 returned error can't find the container with id 45065e98edf0b5d7e6425deda71445fcfe00613bd6726ff85229c0ec43448805
Jul 27 21:52:42.901031 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.900979    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx" event={"ID":"9e582ba3-399e-476f-9483-b5df79cc2ec2","Type":"ContainerStarted","Data":"b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1"}
Jul 27 21:52:42.901180 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.901159    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:42.901276 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.901257    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx" event={"ID":"9e582ba3-399e-476f-9483-b5df79cc2ec2","Type":"ContainerStarted","Data":"27eccb3a5baa1d3d5e0cb3fd152163bd4db5ea268ef910e48e2519b7da749079"}
Jul 27 21:52:42.902792 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.902765    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" event={"ID":"c6727b17-30a0-4170-b4e8-e04928ef327e","Type":"ContainerStarted","Data":"e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9"}
Jul 27 21:52:42.902908 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.902808    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" event={"ID":"c6727b17-30a0-4170-b4e8-e04928ef327e","Type":"ContainerStarted","Data":"ca13ecbb1a72f97d1ee598f7df17c4fd8296968439b128d7657b082fb7bf1f37"}
Jul 27 21:52:42.902908 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.902830    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:42.904384 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.904356    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2" event={"ID":"dc7cf966-3fa4-4019-a423-66ebf89bcff6","Type":"ContainerStarted","Data":"1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2"}
Jul 27 21:52:42.904468 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.904394    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2" event={"ID":"dc7cf966-3fa4-4019-a423-66ebf89bcff6","Type":"ContainerStarted","Data":"45065e98edf0b5d7e6425deda71445fcfe00613bd6726ff85229c0ec43448805"}
Jul 27 21:52:42.904468 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.904422    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:42.917339 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.917286    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx" podStartSLOduration=0.917264139 podStartE2EDuration="917.264139ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:52:42.915963411 +0000 UTC m=+3306.219208181" watchObservedRunningTime="2026-07-27 21:52:42.917264139 +0000 UTC m=+3306.220508905"
Jul 27 21:52:42.935931 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.935875    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" podStartSLOduration=1.491874162 podStartE2EDuration="1.935855309s" totalImagesPullingTime="443.981147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:52:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:52:42.934062509 +0000 UTC m=+3306.237307275" watchObservedRunningTime="2026-07-27 21:52:42.935855309 +0000 UTC m=+3306.239100077"
Jul 27 21:52:42.950545 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:42.950492    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2" podStartSLOduration=0.950471833 podStartE2EDuration="950.471833ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:52:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:52:42.949862954 +0000 UTC m=+3306.253107720" watchObservedRunningTime="2026-07-27 21:52:42.950471833 +0000 UTC m=+3306.253716598"
Jul 27 21:52:48.914947 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:48.914919    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:52:53.911408 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:53.911375    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:52:53.911875 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:53.911679    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:52:54.555473 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.555439    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"]
Jul 27 21:52:54.557982 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.557959    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:52:54.560475 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.560458    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22s5m\""
Jul 27 21:52:54.566014 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.565950    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"]
Jul 27 21:52:54.644988 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.644954    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b2d2be9-e136-4030-b7ca-ff7f8bb28040-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz\" (UID: \"7b2d2be9-e136-4030-b7ca-ff7f8bb28040\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:52:54.746054 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.746028    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b2d2be9-e136-4030-b7ca-ff7f8bb28040-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz\" (UID: \"7b2d2be9-e136-4030-b7ca-ff7f8bb28040\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:52:54.746643 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.746625    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b2d2be9-e136-4030-b7ca-ff7f8bb28040-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz\" (UID: \"7b2d2be9-e136-4030-b7ca-ff7f8bb28040\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:52:54.870336 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:54.870278    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:52:55.003700 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:55.003669    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"]
Jul 27 21:52:55.004161 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:52:55.004133    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2d2be9_e136_4030_b7ca_ff7f8bb28040.slice/crio-83be071f995931f7ac571ae5b2c1a29e69bf6a119d812ae2c69a04e535b16277 WatchSource:0}: Error finding container 83be071f995931f7ac571ae5b2c1a29e69bf6a119d812ae2c69a04e535b16277: Status 404 returned error can't find the container with id 83be071f995931f7ac571ae5b2c1a29e69bf6a119d812ae2c69a04e535b16277
Jul 27 21:52:55.966924 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:55.966847    2552 generic.go:356] "Generic (PLEG): container finished" podID="7b2d2be9-e136-4030-b7ca-ff7f8bb28040" containerID="28f6d9cef47b2b0789a22eeb3f6bb22e57847a9b3e26c2e2f284eef7bfa9b088" exitCode=0
Jul 27 21:52:55.967041 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:55.966925    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz" event={"ID":"7b2d2be9-e136-4030-b7ca-ff7f8bb28040","Type":"ContainerDied","Data":"28f6d9cef47b2b0789a22eeb3f6bb22e57847a9b3e26c2e2f284eef7bfa9b088"}
Jul 27 21:52:55.967041 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:55.966978    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz" event={"ID":"7b2d2be9-e136-4030-b7ca-ff7f8bb28040","Type":"ContainerStarted","Data":"83be071f995931f7ac571ae5b2c1a29e69bf6a119d812ae2c69a04e535b16277"}
Jul 27 21:52:57.103790 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.103766    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:52:57.161999 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.161969    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b2d2be9-e136-4030-b7ca-ff7f8bb28040-maas-api-service-ca\") pod \"7b2d2be9-e136-4030-b7ca-ff7f8bb28040\" (UID: \"7b2d2be9-e136-4030-b7ca-ff7f8bb28040\") "
Jul 27 21:52:57.162328 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.162306    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b2d2be9-e136-4030-b7ca-ff7f8bb28040-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b2d2be9-e136-4030-b7ca-ff7f8bb28040" (UID: "7b2d2be9-e136-4030-b7ca-ff7f8bb28040"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:52:57.262665 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.262605    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b2d2be9-e136-4030-b7ca-ff7f8bb28040-maas-api-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:52:57.978739 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.978674    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz" event={"ID":"7b2d2be9-e136-4030-b7ca-ff7f8bb28040","Type":"ContainerDied","Data":"83be071f995931f7ac571ae5b2c1a29e69bf6a119d812ae2c69a04e535b16277"}
Jul 27 21:52:57.978909 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.978760    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83be071f995931f7ac571ae5b2c1a29e69bf6a119d812ae2c69a04e535b16277"
Jul 27 21:52:57.978909 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:52:57.978707    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"
Jul 27 21:53:08.610715 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.610679    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"]
Jul 27 21:53:08.611147 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.610946    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" podUID="c6727b17-30a0-4170-b4e8-e04928ef327e" containerName="maas-api" containerID="cri-o://e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9" gracePeriod=30
Jul 27 21:53:08.656570 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.656541    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"]
Jul 27 21:53:08.656824 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.656777    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2" podUID="dc7cf966-3fa4-4019-a423-66ebf89bcff6" containerName="payload-processing" containerID="cri-o://1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2" gracePeriod=30
Jul 27 21:53:08.664258 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.664166    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"]
Jul 27 21:53:08.664258 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.664402    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx" podUID="9e582ba3-399e-476f-9483-b5df79cc2ec2" containerName="payload-pre-processing" containerID="cri-o://b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1" gracePeriod=30
Jul 27 21:53:08.797624 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.797594    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"]
Jul 27 21:53:08.798463 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.798435    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-ad54ee-qq5mz"]
Jul 27 21:53:08.935312 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.935289    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:53:08.946651 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.946630    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:53:08.953587 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.953569    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:53:08.964478 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.964456    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsxlj\" (UniqueName: \"kubernetes.io/projected/dc7cf966-3fa4-4019-a423-66ebf89bcff6-kube-api-access-wsxlj\") pod \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") "
Jul 27 21:53:08.964600 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.964483    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc7cf966-3fa4-4019-a423-66ebf89bcff6-plugins-config-volume\") pod \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\" (UID: \"dc7cf966-3fa4-4019-a423-66ebf89bcff6\") "
Jul 27 21:53:08.964903 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.964881    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc7cf966-3fa4-4019-a423-66ebf89bcff6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc7cf966-3fa4-4019-a423-66ebf89bcff6" (UID: "dc7cf966-3fa4-4019-a423-66ebf89bcff6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:53:08.966663 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:08.966639    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc7cf966-3fa4-4019-a423-66ebf89bcff6-kube-api-access-wsxlj" (OuterVolumeSpecName: "kube-api-access-wsxlj") pod "dc7cf966-3fa4-4019-a423-66ebf89bcff6" (UID: "dc7cf966-3fa4-4019-a423-66ebf89bcff6"). InnerVolumeSpecName "kube-api-access-wsxlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:53:09.033366 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.033333    2552 generic.go:356] "Generic (PLEG): container finished" podID="dc7cf966-3fa4-4019-a423-66ebf89bcff6" containerID="1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2" exitCode=0
Jul 27 21:53:09.033517 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.033378    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"
Jul 27 21:53:09.033517 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.033417    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2" event={"ID":"dc7cf966-3fa4-4019-a423-66ebf89bcff6","Type":"ContainerDied","Data":"1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2"}
Jul 27 21:53:09.033517 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.033466    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2" event={"ID":"dc7cf966-3fa4-4019-a423-66ebf89bcff6","Type":"ContainerDied","Data":"45065e98edf0b5d7e6425deda71445fcfe00613bd6726ff85229c0ec43448805"}
Jul 27 21:53:09.033517 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.033487    2552 scope.go:117] "RemoveContainer" containerID="1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2"
Jul 27 21:53:09.034768 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.034748    2552 generic.go:356] "Generic (PLEG): container finished" podID="9e582ba3-399e-476f-9483-b5df79cc2ec2" containerID="b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1" exitCode=0
Jul 27 21:53:09.034877 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.034800    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"
Jul 27 21:53:09.034877 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.034838    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx" event={"ID":"9e582ba3-399e-476f-9483-b5df79cc2ec2","Type":"ContainerDied","Data":"b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1"}
Jul 27 21:53:09.034993 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.034880    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx" event={"ID":"9e582ba3-399e-476f-9483-b5df79cc2ec2","Type":"ContainerDied","Data":"27eccb3a5baa1d3d5e0cb3fd152163bd4db5ea268ef910e48e2519b7da749079"}
Jul 27 21:53:09.036238 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.036200    2552 generic.go:356] "Generic (PLEG): container finished" podID="c6727b17-30a0-4170-b4e8-e04928ef327e" containerID="e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9" exitCode=0
Jul 27 21:53:09.036348 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.036258    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" event={"ID":"c6727b17-30a0-4170-b4e8-e04928ef327e","Type":"ContainerDied","Data":"e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9"}
Jul 27 21:53:09.036348 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.036296    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" event={"ID":"c6727b17-30a0-4170-b4e8-e04928ef327e","Type":"ContainerDied","Data":"ca13ecbb1a72f97d1ee598f7df17c4fd8296968439b128d7657b082fb7bf1f37"}
Jul 27 21:53:09.036348 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.036326    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"
Jul 27 21:53:09.043098 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.043082    2552 scope.go:117] "RemoveContainer" containerID="1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2"
Jul 27 21:53:09.043386 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:53:09.043363    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2\": container with ID starting with 1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2 not found: ID does not exist" containerID="1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2"
Jul 27 21:53:09.043450 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.043396    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2"} err="failed to get container status \"1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2\": rpc error: code = NotFound desc = could not find container \"1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2\": container with ID starting with 1cbcd587f67d5a54bd28798b2dd23d8e0ca6d9faeeb1afd4ef62adb981cea1a2 not found: ID does not exist"
Jul 27 21:53:09.043450 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.043413    2552 scope.go:117] "RemoveContainer" containerID="b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1"
Jul 27 21:53:09.051104 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.051087    2552 scope.go:117] "RemoveContainer" containerID="b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1"
Jul 27 21:53:09.051356 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:53:09.051333    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1\": container with ID starting with b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1 not found: ID does not exist" containerID="b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1"
Jul 27 21:53:09.051473 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.051362    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1"} err="failed to get container status \"b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1\": rpc error: code = NotFound desc = could not find container \"b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1\": container with ID starting with b474a786387cd9b5ee8d11d8a707420a84f6739ef1b10a825178f25aaab9e6d1 not found: ID does not exist"
Jul 27 21:53:09.051473 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.051383    2552 scope.go:117] "RemoveContainer" containerID="e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9"
Jul 27 21:53:09.054860 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.054840    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"]
Jul 27 21:53:09.059357 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.059338    2552 scope.go:117] "RemoveContainer" containerID="e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9"
Jul 27 21:53:09.059626 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:53:09.059608    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9\": container with ID starting with e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9 not found: ID does not exist" containerID="e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9"
Jul 27 21:53:09.059692 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.059632    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9"} err="failed to get container status \"e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9\": rpc error: code = NotFound desc = could not find container \"e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9\": container with ID starting with e54a925e0611e02da85ac658ececaca98f1a754ca63bd9e7c4d529ed468f27d9 not found: ID does not exist"
Jul 27 21:53:09.059768 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.059754    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-ad54ee-b4dc759b4-bstc2"]
Jul 27 21:53:09.065866 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.065846    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6727b17-30a0-4170-b4e8-e04928ef327e-openshift-service-ca\") pod \"c6727b17-30a0-4170-b4e8-e04928ef327e\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") "
Jul 27 21:53:09.065948 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.065898    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6727b17-30a0-4170-b4e8-e04928ef327e-maas-api-tls\") pod \"c6727b17-30a0-4170-b4e8-e04928ef327e\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") "
Jul 27 21:53:09.066017 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066005    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm74f\" (UniqueName: \"kubernetes.io/projected/c6727b17-30a0-4170-b4e8-e04928ef327e-kube-api-access-wm74f\") pod \"c6727b17-30a0-4170-b4e8-e04928ef327e\" (UID: \"c6727b17-30a0-4170-b4e8-e04928ef327e\") "
Jul 27 21:53:09.066073 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066033    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpnm4\" (UniqueName: \"kubernetes.io/projected/9e582ba3-399e-476f-9483-b5df79cc2ec2-kube-api-access-rpnm4\") pod \"9e582ba3-399e-476f-9483-b5df79cc2ec2\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") "
Jul 27 21:53:09.066126 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066107    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e582ba3-399e-476f-9483-b5df79cc2ec2-plugins-config-volume\") pod \"9e582ba3-399e-476f-9483-b5df79cc2ec2\" (UID: \"9e582ba3-399e-476f-9483-b5df79cc2ec2\") "
Jul 27 21:53:09.066182 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066143    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6727b17-30a0-4170-b4e8-e04928ef327e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c6727b17-30a0-4170-b4e8-e04928ef327e" (UID: "c6727b17-30a0-4170-b4e8-e04928ef327e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:53:09.066367 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066352    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsxlj\" (UniqueName: \"kubernetes.io/projected/dc7cf966-3fa4-4019-a423-66ebf89bcff6-kube-api-access-wsxlj\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.066431 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066372    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc7cf966-3fa4-4019-a423-66ebf89bcff6-plugins-config-volume\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.066431 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066385    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6727b17-30a0-4170-b4e8-e04928ef327e-openshift-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.066546 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.066525    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e582ba3-399e-476f-9483-b5df79cc2ec2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9e582ba3-399e-476f-9483-b5df79cc2ec2" (UID: "9e582ba3-399e-476f-9483-b5df79cc2ec2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:53:09.068211 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.068192    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6727b17-30a0-4170-b4e8-e04928ef327e-kube-api-access-wm74f" (OuterVolumeSpecName: "kube-api-access-wm74f") pod "c6727b17-30a0-4170-b4e8-e04928ef327e" (UID: "c6727b17-30a0-4170-b4e8-e04928ef327e"). InnerVolumeSpecName "kube-api-access-wm74f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:53:09.068680 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.068660    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6727b17-30a0-4170-b4e8-e04928ef327e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c6727b17-30a0-4170-b4e8-e04928ef327e" (UID: "c6727b17-30a0-4170-b4e8-e04928ef327e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:53:09.068728 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.068673    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e582ba3-399e-476f-9483-b5df79cc2ec2-kube-api-access-rpnm4" (OuterVolumeSpecName: "kube-api-access-rpnm4") pod "9e582ba3-399e-476f-9483-b5df79cc2ec2" (UID: "9e582ba3-399e-476f-9483-b5df79cc2ec2"). InnerVolumeSpecName "kube-api-access-rpnm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:53:09.167745 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.167714    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm74f\" (UniqueName: \"kubernetes.io/projected/c6727b17-30a0-4170-b4e8-e04928ef327e-kube-api-access-wm74f\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.167745 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.167746    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpnm4\" (UniqueName: \"kubernetes.io/projected/9e582ba3-399e-476f-9483-b5df79cc2ec2-kube-api-access-rpnm4\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.167921 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.167758    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9e582ba3-399e-476f-9483-b5df79cc2ec2-plugins-config-volume\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.167921 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.167767    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6727b17-30a0-4170-b4e8-e04928ef327e-maas-api-tls\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:09.198853 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.198824    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b2d2be9-e136-4030-b7ca-ff7f8bb28040" path="/var/lib/kubelet/pods/7b2d2be9-e136-4030-b7ca-ff7f8bb28040/volumes"
Jul 27 21:53:09.199176 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.199163    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc7cf966-3fa4-4019-a423-66ebf89bcff6" path="/var/lib/kubelet/pods/dc7cf966-3fa4-4019-a423-66ebf89bcff6/volumes"
Jul 27 21:53:09.349029 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.349000    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"]
Jul 27 21:53:09.353105 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.353083    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-ad54ee-678887c764-rnbhx"]
Jul 27 21:53:09.363170 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.363145    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"]
Jul 27 21:53:09.366251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:09.366216    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5"]
Jul 27 21:53:11.197885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:11.197848    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e582ba3-399e-476f-9483-b5df79cc2ec2" path="/var/lib/kubelet/pods/9e582ba3-399e-476f-9483-b5df79cc2ec2/volumes"
Jul 27 21:53:11.198275 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:11.198260    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6727b17-30a0-4170-b4e8-e04928ef327e" path="/var/lib/kubelet/pods/c6727b17-30a0-4170-b4e8-e04928ef327e/volumes"
Jul 27 21:53:11.911162 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:11.911123    2552 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-ad54ee-7fb7899d8d-w2dx5" podUID="c6727b17-30a0-4170-b4e8-e04928ef327e" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.37:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 27 21:53:19.404010 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.403975    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"]
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404313    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b2d2be9-e136-4030-b7ca-ff7f8bb28040" containerName="revoke-keys"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404324    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b2d2be9-e136-4030-b7ca-ff7f8bb28040" containerName="revoke-keys"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404337    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e582ba3-399e-476f-9483-b5df79cc2ec2" containerName="payload-pre-processing"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404342    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e582ba3-399e-476f-9483-b5df79cc2ec2" containerName="payload-pre-processing"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404352    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc7cf966-3fa4-4019-a423-66ebf89bcff6" containerName="payload-processing"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404358    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc7cf966-3fa4-4019-a423-66ebf89bcff6" containerName="payload-processing"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404372    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6727b17-30a0-4170-b4e8-e04928ef327e" containerName="maas-api"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404378    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6727b17-30a0-4170-b4e8-e04928ef327e" containerName="maas-api"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404428    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc7cf966-3fa4-4019-a423-66ebf89bcff6" containerName="payload-processing"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404438    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b2d2be9-e136-4030-b7ca-ff7f8bb28040" containerName="revoke-keys"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404444    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e582ba3-399e-476f-9483-b5df79cc2ec2" containerName="payload-pre-processing"
Jul 27 21:53:19.404491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.404452    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6727b17-30a0-4170-b4e8-e04928ef327e" containerName="maas-api"
Jul 27 21:53:19.406614 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.406595    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:19.409243 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.409205    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22s5m\""
Jul 27 21:53:19.414904 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.414872    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"]
Jul 27 21:53:19.460824 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.460790    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-262475-kjggr\" (UID: \"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:19.562286 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.562255    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-262475-kjggr\" (UID: \"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:19.562861 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.562838    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-262475-kjggr\" (UID: \"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:19.717491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.717464    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:19.844624 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:19.844603    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"]
Jul 27 21:53:19.846750 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:53:19.846719    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74655b1_fc13_4b7e_99cf_be2a73eb9b1a.slice/crio-adba363a61e636a91d181e167cdce5b4ca13d921c665e021fee68135c2aadc45 WatchSource:0}: Error finding container adba363a61e636a91d181e167cdce5b4ca13d921c665e021fee68135c2aadc45: Status 404 returned error can't find the container with id adba363a61e636a91d181e167cdce5b4ca13d921c665e021fee68135c2aadc45
Jul 27 21:53:20.089609 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:20.089580    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr" event={"ID":"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a","Type":"ContainerStarted","Data":"adba363a61e636a91d181e167cdce5b4ca13d921c665e021fee68135c2aadc45"}
Jul 27 21:53:21.102525 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:21.102485    2552 generic.go:356] "Generic (PLEG): container finished" podID="c74655b1-fc13-4b7e-99cf-be2a73eb9b1a" containerID="700321bae482d25e3538b527abddb2465632b6c59751d7fa4dcdd885b66a8823" exitCode=0
Jul 27 21:53:21.102994 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:21.102577    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr" event={"ID":"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a","Type":"ContainerDied","Data":"700321bae482d25e3538b527abddb2465632b6c59751d7fa4dcdd885b66a8823"}
Jul 27 21:53:22.227786 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:22.227765    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:22.288307 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:22.288282    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a-maas-api-service-ca\") pod \"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a\" (UID: \"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a\") "
Jul 27 21:53:22.288687 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:22.288648    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c74655b1-fc13-4b7e-99cf-be2a73eb9b1a" (UID: "c74655b1-fc13-4b7e-99cf-be2a73eb9b1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:53:22.389769 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:22.389702    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a-maas-api-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:23.113432 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:23.113395    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr" event={"ID":"c74655b1-fc13-4b7e-99cf-be2a73eb9b1a","Type":"ContainerDied","Data":"adba363a61e636a91d181e167cdce5b4ca13d921c665e021fee68135c2aadc45"}
Jul 27 21:53:23.113637 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:23.113452    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adba363a61e636a91d181e167cdce5b4ca13d921c665e021fee68135c2aadc45"
Jul 27 21:53:23.113637 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:23.113401    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"
Jul 27 21:53:40.872416 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:40.872361    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"]
Jul 27 21:53:40.875294 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:40.875268    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-262475-kjggr"]
Jul 27 21:53:41.198142 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:41.198106    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c74655b1-fc13-4b7e-99cf-be2a73eb9b1a" path="/var/lib/kubelet/pods/c74655b1-fc13-4b7e-99cf-be2a73eb9b1a/volumes"
Jul 27 21:53:42.024333 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.024298    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jhw5q"]
Jul 27 21:53:42.024728 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.024640    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c74655b1-fc13-4b7e-99cf-be2a73eb9b1a" containerName="revoke-keys"
Jul 27 21:53:42.024728 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.024650    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c74655b1-fc13-4b7e-99cf-be2a73eb9b1a" containerName="revoke-keys"
Jul 27 21:53:42.024728 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.024707    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c74655b1-fc13-4b7e-99cf-be2a73eb9b1a" containerName="revoke-keys"
Jul 27 21:53:42.029011 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.028995    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:42.031605 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.031588    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6krhz\""
Jul 27 21:53:42.034981 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.034959    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jhw5q"]
Jul 27 21:53:42.165333 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.165300    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcgkc\" (UniqueName: \"kubernetes.io/projected/12308112-3976-470f-8ece-49272a8eeb20-kube-api-access-xcgkc\") pod \"kuadrant-operator-catalog-jhw5q\" (UID: \"12308112-3976-470f-8ece-49272a8eeb20\") " pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:42.266177 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.266148    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcgkc\" (UniqueName: \"kubernetes.io/projected/12308112-3976-470f-8ece-49272a8eeb20-kube-api-access-xcgkc\") pod \"kuadrant-operator-catalog-jhw5q\" (UID: \"12308112-3976-470f-8ece-49272a8eeb20\") " pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:42.274501 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.274432    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcgkc\" (UniqueName: \"kubernetes.io/projected/12308112-3976-470f-8ece-49272a8eeb20-kube-api-access-xcgkc\") pod \"kuadrant-operator-catalog-jhw5q\" (UID: \"12308112-3976-470f-8ece-49272a8eeb20\") " pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:42.339265 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.339219    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:42.471680 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:42.471641    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jhw5q"]
Jul 27 21:53:42.474281 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:53:42.474252    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12308112_3976_470f_8ece_49272a8eeb20.slice/crio-9febb98dfb992e92512f8112cd22e45add11b86f665f273b010298d5d1ac8fa5 WatchSource:0}: Error finding container 9febb98dfb992e92512f8112cd22e45add11b86f665f273b010298d5d1ac8fa5: Status 404 returned error can't find the container with id 9febb98dfb992e92512f8112cd22e45add11b86f665f273b010298d5d1ac8fa5
Jul 27 21:53:43.207933 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:43.207903    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q" event={"ID":"12308112-3976-470f-8ece-49272a8eeb20","Type":"ContainerStarted","Data":"9febb98dfb992e92512f8112cd22e45add11b86f665f273b010298d5d1ac8fa5"}
Jul 27 21:53:45.218756 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:45.218720    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q" event={"ID":"12308112-3976-470f-8ece-49272a8eeb20","Type":"ContainerStarted","Data":"2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb"}
Jul 27 21:53:45.238741 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:45.238688    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q" podStartSLOduration=1.527229169 podStartE2EDuration="3.238673838s" totalImagesPullingTime="1.711444669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:53:45.237344356 +0000 UTC m=+3368.540589122" watchObservedRunningTime="2026-07-27 21:53:45.238673838 +0000 UTC m=+3368.541918603"
Jul 27 21:53:46.266210 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:46.266176    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"]
Jul 27 21:53:52.339448 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:52.339411    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:52.339924 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:52.339488    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:52.361254 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:52.361212    2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:53.274386 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:53.274321    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:53.308252 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:53.308206    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jhw5q"]
Jul 27 21:53:55.263738 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:55.263705    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q" podUID="12308112-3976-470f-8ece-49272a8eeb20" containerName="registry-server" containerID="cri-o://2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb" gracePeriod=2
Jul 27 21:53:55.499445 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:55.499422    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:55.584017 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:55.583946    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcgkc\" (UniqueName: \"kubernetes.io/projected/12308112-3976-470f-8ece-49272a8eeb20-kube-api-access-xcgkc\") pod \"12308112-3976-470f-8ece-49272a8eeb20\" (UID: \"12308112-3976-470f-8ece-49272a8eeb20\") "
Jul 27 21:53:55.586051 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:55.586027    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12308112-3976-470f-8ece-49272a8eeb20-kube-api-access-xcgkc" (OuterVolumeSpecName: "kube-api-access-xcgkc") pod "12308112-3976-470f-8ece-49272a8eeb20" (UID: "12308112-3976-470f-8ece-49272a8eeb20"). InnerVolumeSpecName "kube-api-access-xcgkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:53:55.685152 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:55.685130    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcgkc\" (UniqueName: \"kubernetes.io/projected/12308112-3976-470f-8ece-49272a8eeb20-kube-api-access-xcgkc\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:53:56.269479 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.269445    2552 generic.go:356] "Generic (PLEG): container finished" podID="12308112-3976-470f-8ece-49272a8eeb20" containerID="2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb" exitCode=0
Jul 27 21:53:56.269885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.269490    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q"
Jul 27 21:53:56.269885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.269536    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q" event={"ID":"12308112-3976-470f-8ece-49272a8eeb20","Type":"ContainerDied","Data":"2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb"}
Jul 27 21:53:56.269885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.269578    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jhw5q" event={"ID":"12308112-3976-470f-8ece-49272a8eeb20","Type":"ContainerDied","Data":"9febb98dfb992e92512f8112cd22e45add11b86f665f273b010298d5d1ac8fa5"}
Jul 27 21:53:56.269885 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.269600    2552 scope.go:117] "RemoveContainer" containerID="2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb"
Jul 27 21:53:56.279437 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.279420    2552 scope.go:117] "RemoveContainer" containerID="2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb"
Jul 27 21:53:56.279658 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:53:56.279638    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb\": container with ID starting with 2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb not found: ID does not exist" containerID="2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb"
Jul 27 21:53:56.279706 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.279668    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb"} err="failed to get container status \"2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb\": rpc error: code = NotFound desc = could not find container \"2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb\": container with ID starting with 2941c6a65383a319b5cc45244e51c8a370497c4b2086e60c914e3b0e7ea50bdb not found: ID does not exist"
Jul 27 21:53:56.323453 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.323428    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jhw5q"]
Jul 27 21:53:56.341430 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:56.341408    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jhw5q"]
Jul 27 21:53:57.198453 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:53:57.198421    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12308112-3976-470f-8ece-49272a8eeb20" path="/var/lib/kubelet/pods/12308112-3976-470f-8ece-49272a8eeb20/volumes"
Jul 27 21:54:01.285608 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.285561    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6" podUID="cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" containerName="main" containerID="cri-o://10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8" gracePeriod=45
Jul 27 21:54:01.533896 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.533871    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:54:01.635315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635234    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tmp-dir\") pod \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") "
Jul 27 21:54:01.635315 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635291    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-home\") pod \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") "
Jul 27 21:54:01.635499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635366    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-model-cache\") pod \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") "
Jul 27 21:54:01.635499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635389    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tls-certs\") pod \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") "
Jul 27 21:54:01.635499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635419    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q65s\" (UniqueName: \"kubernetes.io/projected/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-kube-api-access-5q65s\") pod \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") "
Jul 27 21:54:01.635499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635444    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-dshm\") pod \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\" (UID: \"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e\") "
Jul 27 21:54:01.635687 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635571    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" (UID: "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:54:01.635687 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635654    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-home" (OuterVolumeSpecName: "home") pod "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" (UID: "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:54:01.635793 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635768    2552 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tmp-dir\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:01.635793 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635787    2552 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-home\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:01.635893 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.635853    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" (UID: "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:54:01.637848 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.637819    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-dshm" (OuterVolumeSpecName: "dshm") pod "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" (UID: "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 27 21:54:01.637979 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.637924    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-kube-api-access-5q65s" (OuterVolumeSpecName: "kube-api-access-5q65s") pod "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" (UID: "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e"). InnerVolumeSpecName "kube-api-access-5q65s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:54:01.638041 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.637973    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" (UID: "cd551e53-38cf-4ee3-a99a-25a91bfe6c8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:54:01.737288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.737252    2552 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-dshm\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:01.737288 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.737283    2552 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-model-cache\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:01.737484 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.737322    2552 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-tls-certs\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:01.737484 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:01.737337    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q65s\" (UniqueName: \"kubernetes.io/projected/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e-kube-api-access-5q65s\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:02.298685 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.298651    2552 generic.go:356] "Generic (PLEG): container finished" podID="cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" containerID="10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8" exitCode=0
Jul 27 21:54:02.299140 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.298713    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"
Jul 27 21:54:02.299140 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.298724    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6" event={"ID":"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e","Type":"ContainerDied","Data":"10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8"}
Jul 27 21:54:02.299140 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.298755    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6" event={"ID":"cd551e53-38cf-4ee3-a99a-25a91bfe6c8e","Type":"ContainerDied","Data":"e776017f40e5aeafb01aa89cd7476ecc90934702da3e9b4a5df6535f1e64a8a9"}
Jul 27 21:54:02.299140 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.298771    2552 scope.go:117] "RemoveContainer" containerID="10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8"
Jul 27 21:54:02.308031 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.308015    2552 scope.go:117] "RemoveContainer" containerID="10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8"
Jul 27 21:54:02.308306 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:54:02.308289    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8\": container with ID starting with 10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8 not found: ID does not exist" containerID="10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8"
Jul 27 21:54:02.308366 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.308315    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8"} err="failed to get container status \"10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8\": rpc error: code = NotFound desc = could not find container \"10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8\": container with ID starting with 10bf6854e395e4b37ccf35a6100ea9328fdea3d171c914d5d1228739110e6bd8 not found: ID does not exist"
Jul 27 21:54:02.327002 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.326976    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"]
Jul 27 21:54:02.331123 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:02.331102    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-262475/ipp-route-262475-kserve-6d67f7bd67-trqm6"]
Jul 27 21:54:03.197312 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:03.197278    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" path="/var/lib/kubelet/pods/cd551e53-38cf-4ee3-a99a-25a91bfe6c8e/volumes"
Jul 27 21:54:36.370567 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370485    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"]
Jul 27 21:54:36.371022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370883    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12308112-3976-470f-8ece-49272a8eeb20" containerName="registry-server"
Jul 27 21:54:36.371022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370897    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="12308112-3976-470f-8ece-49272a8eeb20" containerName="registry-server"
Jul 27 21:54:36.371022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370916    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" containerName="main"
Jul 27 21:54:36.371022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370921    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" containerName="main"
Jul 27 21:54:36.371022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370979    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="12308112-3976-470f-8ece-49272a8eeb20" containerName="registry-server"
Jul 27 21:54:36.371022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.370987    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd551e53-38cf-4ee3-a99a-25a91bfe6c8e" containerName="main"
Jul 27 21:54:36.373895 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.373879    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:36.377166 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.377145    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22s5m\""
Jul 27 21:54:36.383637 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.383604    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"]
Jul 27 21:54:36.438949 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.438915    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e8e191-9cb8-4d31-bfe2-a89d482ad887-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k\" (UID: \"38e8e191-9cb8-4d31-bfe2-a89d482ad887\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:36.540275 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.540241    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e8e191-9cb8-4d31-bfe2-a89d482ad887-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k\" (UID: \"38e8e191-9cb8-4d31-bfe2-a89d482ad887\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:36.540806 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.540789    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e8e191-9cb8-4d31-bfe2-a89d482ad887-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k\" (UID: \"38e8e191-9cb8-4d31-bfe2-a89d482ad887\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:36.685130 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.685101    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:36.815020 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:36.814989    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"]
Jul 27 21:54:36.818320 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:54:36.818295    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e8e191_9cb8_4d31_bfe2_a89d482ad887.slice/crio-011e76c6c6524100d80ba408a3d92c8f4cc5470969c4b2dec974faf5690a57c0 WatchSource:0}: Error finding container 011e76c6c6524100d80ba408a3d92c8f4cc5470969c4b2dec974faf5690a57c0: Status 404 returned error can't find the container with id 011e76c6c6524100d80ba408a3d92c8f4cc5470969c4b2dec974faf5690a57c0
Jul 27 21:54:37.453871 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:37.453846    2552 generic.go:356] "Generic (PLEG): container finished" podID="38e8e191-9cb8-4d31-bfe2-a89d482ad887" containerID="b6ab2f6bcf90da92b0925f4877cf796f5edd08eff6000b61a3bee633e4d3fc6e" exitCode=0
Jul 27 21:54:37.454152 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:37.453924    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k" event={"ID":"38e8e191-9cb8-4d31-bfe2-a89d482ad887","Type":"ContainerDied","Data":"b6ab2f6bcf90da92b0925f4877cf796f5edd08eff6000b61a3bee633e4d3fc6e"}
Jul 27 21:54:37.454152 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:37.453976    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k" event={"ID":"38e8e191-9cb8-4d31-bfe2-a89d482ad887","Type":"ContainerStarted","Data":"011e76c6c6524100d80ba408a3d92c8f4cc5470969c4b2dec974faf5690a57c0"}
Jul 27 21:54:38.588475 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:38.588452    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:38.660279 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:38.660252    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e8e191-9cb8-4d31-bfe2-a89d482ad887-maas-api-service-ca\") pod \"38e8e191-9cb8-4d31-bfe2-a89d482ad887\" (UID: \"38e8e191-9cb8-4d31-bfe2-a89d482ad887\") "
Jul 27 21:54:38.660590 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:38.660565    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e8e191-9cb8-4d31-bfe2-a89d482ad887-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38e8e191-9cb8-4d31-bfe2-a89d482ad887" (UID: "38e8e191-9cb8-4d31-bfe2-a89d482ad887"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:54:38.761555 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:38.761487    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38e8e191-9cb8-4d31-bfe2-a89d482ad887-maas-api-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:39.465030 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:39.464950    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k" event={"ID":"38e8e191-9cb8-4d31-bfe2-a89d482ad887","Type":"ContainerDied","Data":"011e76c6c6524100d80ba408a3d92c8f4cc5470969c4b2dec974faf5690a57c0"}
Jul 27 21:54:39.465030 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:39.464979    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"
Jul 27 21:54:39.465030 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:39.464998    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="011e76c6c6524100d80ba408a3d92c8f4cc5470969c4b2dec974faf5690a57c0"
Jul 27 21:54:47.862827 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:47.862786    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"]
Jul 27 21:54:47.863398 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:47.863103    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds" podUID="48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" containerName="maas-api" containerID="cri-o://acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb" gracePeriod=30
Jul 27 21:54:47.947079 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:47.947050    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"]
Jul 27 21:54:47.947284 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:47.947256    2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6" podUID="ddc425a3-6406-4ef4-929b-da752821b461" containerName="payload-pre-processing" containerID="cri-o://7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929" gracePeriod=30
Jul 27 21:54:48.146507 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.146478    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"]
Jul 27 21:54:48.150548 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.150519    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c3af52-29h2k"]
Jul 27 21:54:48.188654 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.188632    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:54:48.215699 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.215670    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:54:48.241281 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.241255    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-maas-api-tls\") pod \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") "
Jul 27 21:54:48.241440 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.241313    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvn7l\" (UniqueName: \"kubernetes.io/projected/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-kube-api-access-xvn7l\") pod \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") "
Jul 27 21:54:48.241440 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.241357    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-openshift-service-ca\") pod \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\" (UID: \"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d\") "
Jul 27 21:54:48.241826 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.241745    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" (UID: "48d94ec0-26a7-43e1-8cb2-3774a67bbe7d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:54:48.242615 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.242576    2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-openshift-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:48.243954 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.243930    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" (UID: "48d94ec0-26a7-43e1-8cb2-3774a67bbe7d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 27 21:54:48.244502 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.244482    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-kube-api-access-xvn7l" (OuterVolumeSpecName: "kube-api-access-xvn7l") pod "48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" (UID: "48d94ec0-26a7-43e1-8cb2-3774a67bbe7d"). InnerVolumeSpecName "kube-api-access-xvn7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:54:48.343233 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.343205    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdrd7\" (UniqueName: \"kubernetes.io/projected/ddc425a3-6406-4ef4-929b-da752821b461-kube-api-access-gdrd7\") pod \"ddc425a3-6406-4ef4-929b-da752821b461\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") "
Jul 27 21:54:48.343373 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.343282    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddc425a3-6406-4ef4-929b-da752821b461-plugins-config-volume\") pod \"ddc425a3-6406-4ef4-929b-da752821b461\" (UID: \"ddc425a3-6406-4ef4-929b-da752821b461\") "
Jul 27 21:54:48.343530 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.343514    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-maas-api-tls\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:48.343567 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.343538    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvn7l\" (UniqueName: \"kubernetes.io/projected/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d-kube-api-access-xvn7l\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:48.343657 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.343641    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddc425a3-6406-4ef4-929b-da752821b461-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ddc425a3-6406-4ef4-929b-da752821b461" (UID: "ddc425a3-6406-4ef4-929b-da752821b461"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:54:48.345290 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.345271    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc425a3-6406-4ef4-929b-da752821b461-kube-api-access-gdrd7" (OuterVolumeSpecName: "kube-api-access-gdrd7") pod "ddc425a3-6406-4ef4-929b-da752821b461" (UID: "ddc425a3-6406-4ef4-929b-da752821b461"). InnerVolumeSpecName "kube-api-access-gdrd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 27 21:54:48.444308 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.444284    2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdrd7\" (UniqueName: \"kubernetes.io/projected/ddc425a3-6406-4ef4-929b-da752821b461-kube-api-access-gdrd7\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:48.444308 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.444306    2552 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ddc425a3-6406-4ef4-929b-da752821b461-plugins-config-volume\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:54:48.507419 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.507381    2552 generic.go:356] "Generic (PLEG): container finished" podID="48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" containerID="acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb" exitCode=0
Jul 27 21:54:48.507558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.507440    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"
Jul 27 21:54:48.507558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.507476    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds" event={"ID":"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d","Type":"ContainerDied","Data":"acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb"}
Jul 27 21:54:48.507558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.507522    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds" event={"ID":"48d94ec0-26a7-43e1-8cb2-3774a67bbe7d","Type":"ContainerDied","Data":"961bdccaaa002e59c4cb42df745840fbe62ecb466f3d31cc3cf71d0431005d2d"}
Jul 27 21:54:48.507558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.507551    2552 scope.go:117] "RemoveContainer" containerID="acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb"
Jul 27 21:54:48.508880 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.508859    2552 generic.go:356] "Generic (PLEG): container finished" podID="ddc425a3-6406-4ef4-929b-da752821b461" containerID="7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929" exitCode=0
Jul 27 21:54:48.508987 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.508901    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"
Jul 27 21:54:48.508987 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.508904    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6" event={"ID":"ddc425a3-6406-4ef4-929b-da752821b461","Type":"ContainerDied","Data":"7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929"}
Jul 27 21:54:48.508987 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.508938    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6" event={"ID":"ddc425a3-6406-4ef4-929b-da752821b461","Type":"ContainerDied","Data":"e9f9f31147ac6900105db1dc45ecb53756ad24a3714e9013519fde0247acf9fb"}
Jul 27 21:54:48.516521 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.516335    2552 scope.go:117] "RemoveContainer" containerID="acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb"
Jul 27 21:54:48.516643 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:54:48.516619    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb\": container with ID starting with acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb not found: ID does not exist" containerID="acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb"
Jul 27 21:54:48.516708 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.516657    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb"} err="failed to get container status \"acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb\": rpc error: code = NotFound desc = could not find container \"acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb\": container with ID starting with acb64b5f53b9f5838bd4e98eff7472861db2b0d4f1af35161ce499c910f1edcb not found: ID does not exist"
Jul 27 21:54:48.516708 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.516682    2552 scope.go:117] "RemoveContainer" containerID="7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929"
Jul 27 21:54:48.524324 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.524308    2552 scope.go:117] "RemoveContainer" containerID="7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929"
Jul 27 21:54:48.524547 ip-10-0-140-33 kubenswrapper[2552]: E0727 21:54:48.524531    2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929\": container with ID starting with 7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929 not found: ID does not exist" containerID="7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929"
Jul 27 21:54:48.524621 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.524557    2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929"} err="failed to get container status \"7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929\": rpc error: code = NotFound desc = could not find container \"7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929\": container with ID starting with 7c26b460ef270bdfe5560010f68f3970136551a77a44b53de42df2b163ade929 not found: ID does not exist"
Jul 27 21:54:48.535980 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.535956    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"]
Jul 27 21:54:48.541452 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.541432    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c3af52-5b7b8467b6-657ds"]
Jul 27 21:54:48.554746 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.554726    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"]
Jul 27 21:54:48.561253 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:48.561216    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c3af52-66c64d998d-qn6b6"]
Jul 27 21:54:49.197220 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:49.197178    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38e8e191-9cb8-4d31-bfe2-a89d482ad887" path="/var/lib/kubelet/pods/38e8e191-9cb8-4d31-bfe2-a89d482ad887/volumes"
Jul 27 21:54:49.197762 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:49.197741    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" path="/var/lib/kubelet/pods/48d94ec0-26a7-43e1-8cb2-3774a67bbe7d/volumes"
Jul 27 21:54:49.198106 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:49.198089    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc425a3-6406-4ef4-929b-da752821b461" path="/var/lib/kubelet/pods/ddc425a3-6406-4ef4-929b-da752821b461/volumes"
Jul 27 21:54:56.776022 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.775994    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"]
Jul 27 21:54:56.776402 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776336    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e8e191-9cb8-4d31-bfe2-a89d482ad887" containerName="revoke-keys"
Jul 27 21:54:56.776402 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776349    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e8e191-9cb8-4d31-bfe2-a89d482ad887" containerName="revoke-keys"
Jul 27 21:54:56.776402 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776366    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc425a3-6406-4ef4-929b-da752821b461" containerName="payload-pre-processing"
Jul 27 21:54:56.776402 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776372    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc425a3-6406-4ef4-929b-da752821b461" containerName="payload-pre-processing"
Jul 27 21:54:56.776402 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776380    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" containerName="maas-api"
Jul 27 21:54:56.776402 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776386    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" containerName="maas-api"
Jul 27 21:54:56.776591 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776433    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="38e8e191-9cb8-4d31-bfe2-a89d482ad887" containerName="revoke-keys"
Jul 27 21:54:56.776591 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776444    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc425a3-6406-4ef4-929b-da752821b461" containerName="payload-pre-processing"
Jul 27 21:54:56.776591 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.776451    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="48d94ec0-26a7-43e1-8cb2-3774a67bbe7d" containerName="maas-api"
Jul 27 21:54:56.779462 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.779443    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:54:56.791372 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.791355    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22s5m\""
Jul 27 21:54:56.798502 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.798484    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 21:54:56.807558 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.807522    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"]
Jul 27 21:54:56.921017 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:56.920979    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de45521-7bc3-4d2c-a111-dcb89f42cbdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8\" (UID: \"4de45521-7bc3-4d2c-a111-dcb89f42cbdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:54:57.022266 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.022214    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de45521-7bc3-4d2c-a111-dcb89f42cbdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8\" (UID: \"4de45521-7bc3-4d2c-a111-dcb89f42cbdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:54:57.022816 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.022800    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de45521-7bc3-4d2c-a111-dcb89f42cbdb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8\" (UID: \"4de45521-7bc3-4d2c-a111-dcb89f42cbdb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:54:57.089139 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.089068    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:54:57.253522 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.253501    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"]
Jul 27 21:54:57.255764 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:54:57.255736    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de45521_7bc3_4d2c_a111_dcb89f42cbdb.slice/crio-33e68467c2cba4fbd89b16cee5e3ec96a3b324d3deb948c6531733f6446c0112 WatchSource:0}: Error finding container 33e68467c2cba4fbd89b16cee5e3ec96a3b324d3deb948c6531733f6446c0112: Status 404 returned error can't find the container with id 33e68467c2cba4fbd89b16cee5e3ec96a3b324d3deb948c6531733f6446c0112
Jul 27 21:54:57.551597 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.551572    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8" event={"ID":"4de45521-7bc3-4d2c-a111-dcb89f42cbdb","Type":"ContainerStarted","Data":"7d3b0123118d368799710c33ef7b285555509da05d5ff53059c7cfd11ae62e24"}
Jul 27 21:54:57.551708 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.551634    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8" event={"ID":"4de45521-7bc3-4d2c-a111-dcb89f42cbdb","Type":"ContainerStarted","Data":"33e68467c2cba4fbd89b16cee5e3ec96a3b324d3deb948c6531733f6446c0112"}
Jul 27 21:54:57.597301 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:57.597254    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8" podStartSLOduration=1.597239122 podStartE2EDuration="1.597239122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:54:57.595870387 +0000 UTC m=+3440.899115152" watchObservedRunningTime="2026-07-27 21:54:57.597239122 +0000 UTC m=+3440.900483881"
Jul 27 21:54:58.558422 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:58.558382    2552 generic.go:356] "Generic (PLEG): container finished" podID="4de45521-7bc3-4d2c-a111-dcb89f42cbdb" containerID="7d3b0123118d368799710c33ef7b285555509da05d5ff53059c7cfd11ae62e24" exitCode=0
Jul 27 21:54:58.558870 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:58.558478    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8" event={"ID":"4de45521-7bc3-4d2c-a111-dcb89f42cbdb","Type":"ContainerDied","Data":"7d3b0123118d368799710c33ef7b285555509da05d5ff53059c7cfd11ae62e24"}
Jul 27 21:54:59.688856 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:59.688831    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:54:59.848592 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:59.848514    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de45521-7bc3-4d2c-a111-dcb89f42cbdb-maas-api-service-ca\") pod \"4de45521-7bc3-4d2c-a111-dcb89f42cbdb\" (UID: \"4de45521-7bc3-4d2c-a111-dcb89f42cbdb\") "
Jul 27 21:54:59.848843 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:59.848818    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de45521-7bc3-4d2c-a111-dcb89f42cbdb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4de45521-7bc3-4d2c-a111-dcb89f42cbdb" (UID: "4de45521-7bc3-4d2c-a111-dcb89f42cbdb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:54:59.950090 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:54:59.950059    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4de45521-7bc3-4d2c-a111-dcb89f42cbdb-maas-api-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:55:00.567723 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:00.567694    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8" event={"ID":"4de45521-7bc3-4d2c-a111-dcb89f42cbdb","Type":"ContainerDied","Data":"33e68467c2cba4fbd89b16cee5e3ec96a3b324d3deb948c6531733f6446c0112"}
Jul 27 21:55:00.567723 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:00.567721    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"
Jul 27 21:55:00.567723 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:00.567742    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33e68467c2cba4fbd89b16cee5e3ec96a3b324d3deb948c6531733f6446c0112"
Jul 27 21:55:08.969800 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:08.969766    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"]
Jul 27 21:55:08.972611 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:08.972585    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3298cd-sssq8"]
Jul 27 21:55:09.197893 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:09.197849    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de45521-7bc3-4d2c-a111-dcb89f42cbdb" path="/var/lib/kubelet/pods/4de45521-7bc3-4d2c-a111-dcb89f42cbdb/volumes"
Jul 27 21:55:16.971854 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.971821    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"]
Jul 27 21:55:16.972241 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.972198    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de45521-7bc3-4d2c-a111-dcb89f42cbdb" containerName="revoke-keys"
Jul 27 21:55:16.972241 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.972213    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de45521-7bc3-4d2c-a111-dcb89f42cbdb" containerName="revoke-keys"
Jul 27 21:55:16.972320 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.972276    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de45521-7bc3-4d2c-a111-dcb89f42cbdb" containerName="revoke-keys"
Jul 27 21:55:16.975039 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.975020    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:16.978950 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.978925    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-22s5m\""
Jul 27 21:55:16.979042 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.979005    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 27 21:55:16.987665 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.987642    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"]
Jul 27 21:55:16.993944 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:16.993918    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f25519-3a45-4df7-83b1-15c158112e5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx\" (UID: \"44f25519-3a45-4df7-83b1-15c158112e5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:17.095379 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.095350    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f25519-3a45-4df7-83b1-15c158112e5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx\" (UID: \"44f25519-3a45-4df7-83b1-15c158112e5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:17.095928 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.095910    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f25519-3a45-4df7-83b1-15c158112e5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx\" (UID: \"44f25519-3a45-4df7-83b1-15c158112e5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:17.285333 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.285236    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:17.429627 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.429474    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"]
Jul 27 21:55:17.431888 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:55:17.431860    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f25519_3a45_4df7_83b1_15c158112e5b.slice/crio-b97d6dedda7e509e01abcb5eb179b756b9ed56beb15de120a6d7e2149ce40c16 WatchSource:0}: Error finding container b97d6dedda7e509e01abcb5eb179b756b9ed56beb15de120a6d7e2149ce40c16: Status 404 returned error can't find the container with id b97d6dedda7e509e01abcb5eb179b756b9ed56beb15de120a6d7e2149ce40c16
Jul 27 21:55:17.650209 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.650175    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx" event={"ID":"44f25519-3a45-4df7-83b1-15c158112e5b","Type":"ContainerStarted","Data":"a9e531e1e4bd7c2be0219d24e4f7c624d23e52b73a538a315660cc04423dec1b"}
Jul 27 21:55:17.650367 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.650260    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx" event={"ID":"44f25519-3a45-4df7-83b1-15c158112e5b","Type":"ContainerStarted","Data":"b97d6dedda7e509e01abcb5eb179b756b9ed56beb15de120a6d7e2149ce40c16"}
Jul 27 21:55:17.670382 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:17.670336    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx" podStartSLOduration=1.670321902 podStartE2EDuration="1.670321902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 21:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:55:17.669128587 +0000 UTC m=+3460.972373353" watchObservedRunningTime="2026-07-27 21:55:17.670321902 +0000 UTC m=+3460.973566668"
Jul 27 21:55:18.656788 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:18.656749    2552 generic.go:356] "Generic (PLEG): container finished" podID="44f25519-3a45-4df7-83b1-15c158112e5b" containerID="a9e531e1e4bd7c2be0219d24e4f7c624d23e52b73a538a315660cc04423dec1b" exitCode=0
Jul 27 21:55:18.657282 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:18.656834    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx" event={"ID":"44f25519-3a45-4df7-83b1-15c158112e5b","Type":"ContainerDied","Data":"a9e531e1e4bd7c2be0219d24e4f7c624d23e52b73a538a315660cc04423dec1b"}
Jul 27 21:55:19.789111 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:19.789082    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:19.820419 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:19.820394    2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f25519-3a45-4df7-83b1-15c158112e5b-maas-api-service-ca\") pod \"44f25519-3a45-4df7-83b1-15c158112e5b\" (UID: \"44f25519-3a45-4df7-83b1-15c158112e5b\") "
Jul 27 21:55:19.820812 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:19.820786    2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44f25519-3a45-4df7-83b1-15c158112e5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "44f25519-3a45-4df7-83b1-15c158112e5b" (UID: "44f25519-3a45-4df7-83b1-15c158112e5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 27 21:55:19.921896 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:19.921834    2552 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/44f25519-3a45-4df7-83b1-15c158112e5b-maas-api-service-ca\") on node \"ip-10-0-140-33.ec2.internal\" DevicePath \"\""
Jul 27 21:55:20.670471 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:20.670438    2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"
Jul 27 21:55:20.670471 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:20.670461    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx" event={"ID":"44f25519-3a45-4df7-83b1-15c158112e5b","Type":"ContainerDied","Data":"b97d6dedda7e509e01abcb5eb179b756b9ed56beb15de120a6d7e2149ce40c16"}
Jul 27 21:55:20.670676 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:20.670509    2552 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b97d6dedda7e509e01abcb5eb179b756b9ed56beb15de120a6d7e2149ce40c16"
Jul 27 21:55:28.470021 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:28.469986    2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"]
Jul 27 21:55:28.472713 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:28.472689    2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-2cf6fc-qx4xx"]
Jul 27 21:55:29.197999 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:29.197960    2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44f25519-3a45-4df7-83b1-15c158112e5b" path="/var/lib/kubelet/pods/44f25519-3a45-4df7-83b1-15c158112e5b/volumes"
Jul 27 21:55:42.208592 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:42.208562    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-798cd98846-6b6nw_ad9c960e-3417-46d6-a4ed-eda9659a0f5d/manager/0.log"
Jul 27 21:55:42.216155 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:42.216133    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-798cd98846-6b6nw_ad9c960e-3417-46d6-a4ed-eda9659a0f5d/copy-manifests/0.log"
Jul 27 21:55:45.112767 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:45.112740    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s4h2g_ba7b0df9-c0f0-4a8e-b548-df60552a0b03/kuadrant-console-plugin/0.log"
Jul 27 21:55:45.373609 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:45.373536    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qzcqd_f3472236-38ab-4c86-b69d-df139f36fee4/manager/0.log"
Jul 27 21:55:46.376410 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:46.376381    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d89c5fb64-sz68b_a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc/kube-auth-proxy/0.log"
Jul 27 21:55:47.286218 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:47.286188    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rrvnq_f3e06410-7430-4a13-a8ac-271a60db6a68/main/0.log"
Jul 27 21:55:47.659236 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:47.659207    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fk5fc5_b951cf7c-f233-4ab7-b0c6-905d6aa1d3a2/main/0.log"
Jul 27 21:55:47.785486 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:47.785458    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-69vxc_e7f4f727-d404-4827-afed-8ffcc427cff5/main/0.log"
Jul 27 21:55:56.696429 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:56.696395    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dp2gb_3851429a-b5b9-4a29-b124-f84984721273/global-pull-secret-syncer/0.log"
Jul 27 21:55:56.818645 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:56.818613    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h7sbg_fcfc6382-3050-4547-a388-ff49bbb472c4/konnectivity-agent/0.log"
Jul 27 21:55:56.902158 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:55:56.902132    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-33.ec2.internal_673a856a2cbcee0e523f1fb250518aab/haproxy/0.log"
Jul 27 21:56:02.347268 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:02.347218    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s4h2g_ba7b0df9-c0f0-4a8e-b548-df60552a0b03/kuadrant-console-plugin/0.log"
Jul 27 21:56:02.609785 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:02.609681    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qzcqd_f3472236-38ab-4c86-b69d-df139f36fee4/manager/0.log"
Jul 27 21:56:04.635374 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.635344    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-877nb_2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6/kube-state-metrics/0.log"
Jul 27 21:56:04.656411 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.656385    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-877nb_2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6/kube-rbac-proxy-main/0.log"
Jul 27 21:56:04.679531 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.679513    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-877nb_2bed2be2-fcfa-4b1a-9f50-2bc6acfa28f6/kube-rbac-proxy-self/0.log"
Jul 27 21:56:04.711038 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.711015    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76c949c4f5-xbct8_77dfb373-b265-4939-a281-653fba1fb080/metrics-server/0.log"
Jul 27 21:56:04.945215 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.945177    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wlbcw_c0e37be3-390d-4583-aed3-bde7e9f16223/node-exporter/0.log"
Jul 27 21:56:04.970251 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.970196    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wlbcw_c0e37be3-390d-4583-aed3-bde7e9f16223/kube-rbac-proxy/0.log"
Jul 27 21:56:04.993863 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:04.993835    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wlbcw_c0e37be3-390d-4583-aed3-bde7e9f16223/init-textfile/0.log"
Jul 27 21:56:05.109300 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.109269    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/prometheus/0.log"
Jul 27 21:56:05.131392 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.131371    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/config-reloader/0.log"
Jul 27 21:56:05.154264 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.154245    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/thanos-sidecar/0.log"
Jul 27 21:56:05.176460 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.176435    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/kube-rbac-proxy-web/0.log"
Jul 27 21:56:05.198499 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.198445    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/kube-rbac-proxy/0.log"
Jul 27 21:56:05.222189 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.222172    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/kube-rbac-proxy-thanos/0.log"
Jul 27 21:56:05.244887 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.244868    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_dc4a7d48-4806-401c-b331-1f1109371f35/init-config-reloader/0.log"
Jul 27 21:56:05.272752 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.272705    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-7fx4h_630071f6-e860-4ba1-88c4-797f5c74c00a/prometheus-operator/0.log"
Jul 27 21:56:05.293760 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:05.293737    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-7fx4h_630071f6-e860-4ba1-88c4-797f5c74c00a/kube-rbac-proxy/0.log"
Jul 27 21:56:06.724813 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.724777    2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"]
Jul 27 21:56:06.725204 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.725114    2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44f25519-3a45-4df7-83b1-15c158112e5b" containerName="revoke-keys"
Jul 27 21:56:06.725204 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.725125    2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="44f25519-3a45-4df7-83b1-15c158112e5b" containerName="revoke-keys"
Jul 27 21:56:06.725204 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.725192    2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="44f25519-3a45-4df7-83b1-15c158112e5b" containerName="revoke-keys"
Jul 27 21:56:06.728142 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.728127    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.730548 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.730530    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-km5v5\"/\"kube-root-ca.crt\""
Jul 27 21:56:06.731478 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.731458    2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-km5v5\"/\"openshift-service-ca.crt\""
Jul 27 21:56:06.731478 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.731471    2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-km5v5\"/\"default-dockercfg-fvx26\""
Jul 27 21:56:06.736743 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.736720    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"]
Jul 27 21:56:06.811347 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.811323    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-proc\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.811459 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.811360    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p97tg\" (UniqueName: \"kubernetes.io/projected/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-kube-api-access-p97tg\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.811459 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.811383    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-lib-modules\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.811566 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.811506    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-podres\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.811566 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.811530    2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-sys\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912348 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912324    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-podres\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912500 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912353    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-sys\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912500 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912371    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-proc\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912500 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912398    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p97tg\" (UniqueName: \"kubernetes.io/projected/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-kube-api-access-p97tg\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912500 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912420    2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-lib-modules\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912500 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912457    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-sys\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912500 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912474    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-proc\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912731 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912504    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-podres\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.912731 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.912569    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-lib-modules\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:06.923208 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:06.923183    2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p97tg\" (UniqueName: \"kubernetes.io/projected/cb3663ea-7da7-4ea4-8bab-f60b04ad617f-kube-api-access-p97tg\") pod \"perf-node-gather-daemonset-tcnhc\" (UID: \"cb3663ea-7da7-4ea4-8bab-f60b04ad617f\") " pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:07.038535 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:07.038472    2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:07.165731 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:07.165702    2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"]
Jul 27 21:56:07.167805 ip-10-0-140-33 kubenswrapper[2552]: W0727 21:56:07.167767    2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcb3663ea_7da7_4ea4_8bab_f60b04ad617f.slice/crio-136416225d8581ab16e290393471ece879b0a60234dadc4cca75711e5c26912d WatchSource:0}: Error finding container 136416225d8581ab16e290393471ece879b0a60234dadc4cca75711e5c26912d: Status 404 returned error can't find the container with id 136416225d8581ab16e290393471ece879b0a60234dadc4cca75711e5c26912d
Jul 27 21:56:07.884613 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:07.884580    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc" event={"ID":"cb3663ea-7da7-4ea4-8bab-f60b04ad617f","Type":"ContainerStarted","Data":"ec7605b78b0f9375a7c2752b8a03e38e748ba5554af4bd8779abe3b3c8fca564"}
Jul 27 21:56:07.885014 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:07.884629    2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc" event={"ID":"cb3663ea-7da7-4ea4-8bab-f60b04ad617f","Type":"ContainerStarted","Data":"136416225d8581ab16e290393471ece879b0a60234dadc4cca75711e5c26912d"}
Jul 27 21:56:07.885014 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:07.884730    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:07.902107 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:07.902062    2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc" podStartSLOduration=1.902049288 podStartE2EDuration="1.902049288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 21:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 21:56:07.900704513 +0000 UTC m=+3511.203949279" watchObservedRunningTime="2026-07-27 21:56:07.902049288 +0000 UTC m=+3511.205294052"
Jul 27 21:56:09.599984 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:09.599953    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mfplf_a5b0321d-1d87-409a-9277-a7002c2f80b8/dns/0.log"
Jul 27 21:56:09.620390 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:09.620369    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mfplf_a5b0321d-1d87-409a-9277-a7002c2f80b8/kube-rbac-proxy/0.log"
Jul 27 21:56:09.690185 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:09.690159    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dc22d_e2d0da7f-e17f-4522-bc6d-9d57c51f2d06/dns-node-resolver/0.log"
Jul 27 21:56:10.279621 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:10.279592    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qnt7r_93ced636-3c9e-428d-80e8-acb5ea71c7d2/node-ca/0.log"
Jul 27 21:56:11.428021 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:11.427989    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d89c5fb64-sz68b_a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc/kube-auth-proxy/0.log"
Jul 27 21:56:12.395946 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:12.395916    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bkp2g_7c8704a1-3d26-44a4-bb4c-0659e2702945/serve-healthcheck-canary/0.log"
Jul 27 21:56:13.077633 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:13.077609    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f998m_67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2/kube-rbac-proxy/0.log"
Jul 27 21:56:13.098612 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:13.098590    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f998m_67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2/exporter/0.log"
Jul 27 21:56:13.119494 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:13.119473    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f998m_67afd2fe-2c35-4d49-bfe5-bdd53a9a85b2/extractor/0.log"
Jul 27 21:56:13.900166 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:13.900141    2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-km5v5/perf-node-gather-daemonset-tcnhc"
Jul 27 21:56:15.970844 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:15.970816    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-798cd98846-6b6nw_ad9c960e-3417-46d6-a4ed-eda9659a0f5d/manager/0.log"
Jul 27 21:56:15.993470 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:15.993445    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-798cd98846-6b6nw_ad9c960e-3417-46d6-a4ed-eda9659a0f5d/copy-manifests/0.log"
Jul 27 21:56:17.263898 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:17.263876    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d89c5fb64-sz68b_a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc/kube-auth-proxy/0.log"
Jul 27 21:56:17.574528 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:17.574473    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-75c6cc9f98-7ct2k_3bf83e4d-b229-43e3-b485-ffd6e175a928/manager/0.log"
Jul 27 21:56:18.984487 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:18.984461    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d89c5fb64-sz68b_a2d109f8-d8bf-4a7c-b31d-9c4606cbdfcc/kube-auth-proxy/0.log"
Jul 27 21:56:22.540912 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:22.540882    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-rvks2_cafab56b-d0f9-4ae1-871f-578693aafde7/migrator/0.log"
Jul 27 21:56:22.564054 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:22.564034    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-rvks2_cafab56b-d0f9-4ae1-871f-578693aafde7/graceful-termination/0.log"
Jul 27 21:56:24.287407 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.287379    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/kube-multus-additional-cni-plugins/0.log"
Jul 27 21:56:24.309466 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.309444    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/egress-router-binary-copy/0.log"
Jul 27 21:56:24.346627 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.346606    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/cni-plugins/0.log"
Jul 27 21:56:24.384505 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.384485    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/bond-cni-plugin/0.log"
Jul 27 21:56:24.425475 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.425454    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/routeoverride-cni/0.log"
Jul 27 21:56:24.449774 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.449755    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/whereabouts-cni-bincopy/0.log"
Jul 27 21:56:24.473822 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.473795    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w6zkl_cd832b50-8ccc-4380-8ce0-4fe42ab5a939/whereabouts-cni/0.log"
Jul 27 21:56:24.508419 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.508400    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-c7vfj_67a066b6-b4af-49df-bc8b-5ae94c66ac84/kube-multus/0.log"
Jul 27 21:56:24.625491 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.625422    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cgq4s_0d1c14b4-7aec-4c81-98c5-55841ddfa641/network-metrics-daemon/0.log"
Jul 27 21:56:24.647138 ip-10-0-140-33 kubenswrapper[2552]: I0727 21:56:24.647116    2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cgq4s_0d1c14b4-7aec-4c81-98c5-55841ddfa641/kube-rbac-proxy/0.log"